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FO 12462 US 



A Remote Control Apparatus And 



A System Therefor 



BACKGROUND OF THE INVENTION 
Field of the Invention 

The present invention relates to an information 
processing apparatus that can operate various devices 
indirectly by directly operating a remote controller, 
or another device connected with the remote controller, 
and a system therefor. 
Related Background Art 

A remote controller for exercising operational 
control of an apparatus from a remote location is 
employed primarily for home electric appliances, such 
as video recorders and air conditioners . 

In addition, remote controllers that are used for 
televisions and video recorders, and remote controllers 
that are compatible with a plurality of television 
model s have appeared . 

In order to operate a specific model by using a 
conventional remote controller that is compatible with 
a plurality of models, an operator must change the 
setup in the remote controller. 

Thus, as the number of compatible models has 
increased, setup operations have become so complicated 
that operators must accurately identify models that are 
to be controlled and must then refer to operation 
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manuals to set: up remote controllers, so that the 
likelihood is great that operators will mistake when 
they perform setups. And when the models that are to 
be controlled are changed frequently, the setup must be 
5 repeated often and usability will be deteriorated. 

The devices that are to be controlled by such 
remote controllers are limited to those that are 
located that they are inside the effective range of the 
control signals emitted by the remote controllers. 

10 

SUMMARY OF THE INVENTION 

To resolve the above shortcomings, it is one 
objective of the present invention to provide an 
information processing apparatus that can easily 

15 control a variety of devices, and a system therefor. 

It is another objective of the present invention 
to provide a remote control apparatus that can easily 
cope with a plurality of devices without a series of 
complicated setups being required of a user and that 

20 can control the devices from a remote location, and a 
system therefor. 

It is an additional objective of the present 
invention to provide a remote control system that can 
control, via a nearby device, a device in a remote 

25 location to which instructions can not be directly 
transmitted . 

It is a further objective of the present invention 
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■to provide an information processing apparatus that 
can, as needed, determine an appropriate process to be 
performed, such as communication with another device, 
and can then perform the process. 
5 It is still another objective of the present 

invention to provide an information processing 
apparatus that can acquire, from another device, 
information that it is to process, and can then process 
the information. 

10 It is a still additional objective of the present 

invention to provide an information processing 
apparatus that can transmit information, such as an 
instruction that is the result obtained from a process 
it performed, to another device for storage at that 

15 device. 

It is a still further objective of the present 
invention to provide an information processing 
apparatus that can control another device by employing 
an operation unit that controls the information 

20 processing apparatus. 

The other objective of the present invention is to 
provide an information processing apparatus that can 
flexibly cope with changes in the operational 
procedures . 

25 According to one aspect, the present invention 

which achieves these objectives relates to a remote 
control apparatus comprising: 
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reception means for receiving information 
concerning a device to be controlled; 

validation means for validating operating 
procedures for the device to be controlled; and 
5 transmission control means for transmitting an 

operating command to the device based on the operating 
procedures that are validated. 

According to another aspect, the present invention 
which achieves these objectives relates to a remote 
10 system comprising: 

identification means for identifying a device to 
be controlled by a remote control apparatus; and 

transmission control means for transmitting to the 
remote control apparatus information concerning a 
15 device that is identified, 

wherein the remote control apparatus transmits an 
operating command to the device in accordance with 
operating procedures based on the information that is 
received. 

20 According to yet another aspect, the present 

invention which achieves these objectives relates to a 
remote control apparatus comprising: 

input means for receiving an instruction; 
processing means for performing processing in 
25 accordance with the instruction input by the input 
means ; 

communication means for communicating with another 



device; and 

communication control means for permitting the 
communication means to transmit an operation 
instruction, which has been input for another device by 
the input means, to the device to be controlled 
according to the instruction. 

According to still another aspect, the present 
invention which achieves these objectives relates to an 
information processing apparatus comprising: 

determination means for specifying a process to be 
performed; and 

processing means for performing the process 
specified by the determination means. 

According to one more aspect, the present 
invention which achieves these objectives relates to an 
information processing apparatus comprising: 

communication means for communicating with an 
external device; 

processing means for processing information from 
the external device that is received through the 
communication means ; 

acquisition means for acquiring information from 
the external device through the communication means; 
and 

control means for permitting the processing means 
to process the information acquired by the acquisition 
means . 
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According to yet: one more aspect, the present 
invention which achieves these objectives relates to an 
information processing apparatus comprising: 

communication means for communicating with an 
5 external device; 

processing means for processing information from 
the external device that is received through the 
communication means ; 

transmission control means for transmitting to the 
10 external device through the communication means the 

information that is processed by the processing means; 
and 

control means for storing in the external device 
the information that has been transmitted to the 
15 external device through the communication means. 

According to still one more aspect, the present 
invention which achieves these objectives relates to an 
information processing apparatus comprising: 

operating means for performing an operation; 
20 input means for inputting information; 

processing means for, based on the operation, 
processing the information input by the input means; 

output means for outputting results obtained by 
the processing means; and 
25 control means for controlling another device based 

on an operation for the device performed by the 
operation means . 
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According to an additional aspect, the present 
invention which achieves these objectives relates to an 
information processing method comprising: 

a determination step of specifying a process to be 
5 performed by an information processing apparatus; and 

a processing step of performing the process 
specified at the determination step. 

According to yet an additional aspect, the present 
invention which achieves these objectives relates to an 
10 information processing method comprising: 

a processing step of processing information that 
is received from an external device through a 
communication unit, which is used to communicate with 
the external device; 
15 an acquisition step of acquiring information 

through the communication unit from the external 
device; and 

a control step of, at the processing step, 
processing the information that is acquired at the 
20 acquisition step. 

According to still another additional aspect, the 
present invention which achieves these objectives 
relates to an information processing method comprising: 

a processing step of processing information; 
25 a transmission step of transmitting the 

information that has been processed at the processing 
step to an external device through a communication 
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unit, which is used to communicate with the external 
device; and 

a control step of storing in the external device 
the information that has been transmitted through the 
5 communication unit to the external device. 

According to a further aspect, the present 
invention which achieves these objectives relates to an 
information processing method, for an information 
processing apparatus that has an operating unit for 
10 performing processing, comprising: 

an input step of inputting information; 

a processing step of processing, based on the 
operation, the information input at the input step; 

an output step of outputting results obtained at 
15 the processing step; and 

a control step of controlling another device based 
on an operation performed for the device by the 
operation unit . 

According to a still further aspect, the present 
20 invention which achieves these objectives relates to a 
computer-readable storage medium used to store an 
information processing program for controlling a 
computer for processing of information, the program 
comprising codes for enabling the computer to perform: 
25 a determination step of specifying a process to be 

performed by an information processing apparatus; and 

a processing step of performing the process 
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specified at the determination step. 

According to one more additional aspect, the 
present invention which achieves these objectives 
relates to a computer-readable storage medium used to 
5 store an information processing program for controlling 
a computer for processing of information, the program 
comprising codes for enabling the computer to perform: 

a processing step of processing information that 
is received from an external device through a 
10 communication unit, which is used to communicate with 
the external device; 

an acquisition step of acquiring information 
through the communication unit from the external 
device; and 

15 a control step of, at the processing step, 

processing the information that is acquired at the 
acquisition step. 

According to yet one more additional aspect, the 
present invention which achieves these objectives 

20 relates to a computer-readable storage medium used to 

store an information processing program for controlling 
a computer for processing of information, the program 
comprising codes for enabling the computer to perform: 
a processing step of processing information; 

25 a transmission step of transmitting the 

information that has been processed at the processing 
step to an external device through a communication 
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unit, which is used to communicate with the external 
device; and 

a control step of storing in the external device 
the information that has been transmitted through the 
communication unit to the external device. 

According to still one more additional aspect, the 
present invention which achieves these objectives 
relates to a computer-readable storage medium used to 
store an information processing program for controlling 
a computer for processing of information, the program 
comprising codes for enabling the computer to perform: 

an input step of inputting information; 

a processing step of processing, based on the 
operation, the information input at the input step; 

an output step of outputting results obtained at 
the processing step; and 

a control step of controlling another device based 
on an operation performed for the device by the 
operation unit . 

Other objectives and advantages besides those 
discussed above will become apparent to those skilled 
in the art during the course of the description of the 
preferred embodiments of the invention that follows. 
In the description, reference is made to accompanying 
drawings, which form a part thereof, and which 
illustrate an example of the invention. Such example, 
however, is not exhaustive of the various embodiments 
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of the invention, and therefore reference is made to 
the claims that follow the description for determining 
the scope of the invention* 

5 BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a block diagram illustrating the 
arrangement of an information processing system 
according to a first embodiment of the present 
invention; 

10 Fig. 2 is a diagram illustrating the functional 

arrangement according to the first embodiment; 

Fig. 3 is a diagram illustrating the hardware 

arrangement of the individual units according to the 

first embodiment; 
15 Fig. 4 is a flowchart showing the processing 

performed for the first embodiment; 

Fig. 5 is a diagram showing a process for 

selecting an optimal printer from a plurality of 

printers and performing a printing job with it; 
20 Fig. 6 is a diagram for explaining a case where a 

controller is used; 

Fig. 7 is a diagram showing a case where a 

document is read by a scanner for transmission to a 

specific printer; 
25 Figs. 8, 9 and 10 are diagrams illustrating a case 

where a document is read by a scanner for transmission 

to a specific printer that the scanner can not control; 
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Fig. 11 is a diagram illustrating a case where 
information concerning the state of a printer is 
distributed ; 

Fig. 12 is a flowchart showing the processing 
5 performed an eighth embodiment; 

Fig. 13 is a table containing remarks for 
determining whether or not an advance notice should be 
transmitted in consonance with job types and 
conditions; 

10 Figs. 14A and 14B are respectively diagrams of a 

case where a user is informed in advance of information 
to be filed and a case where unnecessary information is 
abandoned; 

Fig. 15 is a flowchart showing the processing for 
15 a ninth embodiment; 

Fig. 16A is a diagram showing the transmission of 
information for the ninth embodiment; 

Fig. 16B is a diagram showing the transmission of 
information when an apparatus corresponding to the 
20 object of a job is not present; 

Fig. 17 is a diagram showing an example display 
for a window by which an optimal method is proposed to 
a user; 

Fig. 18 is a flowchart showing the detailed 
25 procedure for job execution; 

Fig. 19 is a diagram showing an example of 
classified information to be read; 



- 13 - 





Fig. 


20 


is 


an example job table; 




Fig . 


21 


is 


a diagram showing an example printing 




window; 










Fig. 


22 


is 


an example job table; 


5 


Fig. 


23 


is 


a diagram showing the contents of <fil« 




A>; 










Fig. 


24 


is 


a diagram showing an inquiry window; 




Fig. 


25 


is 


an example job table; 




Fig. 


26 


is 


a diagram showing example scheduling 


10 


data; 










Fig. 


27 


is 


a diagram showing an example user 




query; 










Fig. 


28 


is 


an example job table; 




Fig- 


29 


is 


an example job table; 


15 


Fig. 


30 


is 


a diagram showing an example change in 




the status of 


a 


printer; 




Fig. 


31 


is 


a table showing printer statuses; 




Fig. 


32 


is 


an example job table; 




Fig. 


33 


is 


a diagram showing a query window; 


20 


Fig. 


34 


is 


an example job table; 




Fig. 


35 


is 


a diagram showing an example printer 




change status; 






Fig. 


36 


is 


a table for displaying planning and 




action rules ; 






25 


Fig. 


37 


is 


a table for displaying planning and 




action rules ; 








Fig. 


38 


is 


a diagram showing actions and premise 
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conditions for an object; 

Fig. 39 is a table for example apparatus 
properties; 

Fig. 40 is a table for example apparatus statuses; 
5 Figs. 41A and 41B are diagrams showing states 

where a plurality of apparatuses are connected to a 
network; 

Fig. 42 is a diagram showing a functional 
arrangement for performing the processing for a 
1 0 fourteenth embodiment ; 

Fig. 43 is a main flowchart showing the processing 
for the fourteenth embodiment; 

Fig. 44 is a flowchart showing the processing for 
controlling other apparatuses; 
15 Fig. 45 is a flowchart showing the processing 

performed by one's own apparatus; 

Fig. 46 is a diagram showing an example where a 
printer other than a designated printer is employed for 
output; 

20 Fig. 47 is a diagram showing an example where a 

designated printer transfers a job to another printer; 

Fig. 48 is a diagram showing an example where a 
user's printer transfers a job to a printer at a user's 
destination; 

25 Fig. 49 is a flowchart showing the processing for 

a sixteenth embodiment; 

Fig. 50 is a conceptual diagram showing the 
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processing for the sixteenth embodiment; 

Fig. 51 is a diagram showing transmission of a job 
for a seventeenth embodiment; 

Fig. 52 is a flowchart showing the processing 
5 performed by a print job automatic setup unit; 

Fig. 53 is a table showing printing periods and 
consumption of toner for a plurality of printers for 
printing a single character; 

Fig. 54 is a diagram showing the contents of a 
10 print job and parameters designated by a user; 

Fig. 55 is a flowchart showing the processing 
performed by a print job automatic change unit; 

Fig. 56 is a diagram showing the contents of a 
print job and parameters designated by a user; 
15 Fig. 57 is a diagram showing transmission of a job 

for a nineteenth embodiment; 

Fig. 58A is a diagram for explaining an example 
where parameters are set for a print job by employing a 
natural language; 
20 Fig. 58B is a table showing the meanings of 

character strings in the parameter setup, and request 
items with which to complete the meanings; 

Fig. 59 is a diagram showing an example where 
parameters of a print job are set by using a document; 
25 Fig. 60 is a diagram showing transmission of a job 

for a twenty-first embodiment; 

Fig. 61 is a flowchart showing the processing 
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performed by a print job simulation unit; 

Fig. 62 is a flowchart showing the processing 
performed by a printer system according to a twenty- 
second embodiment ; 
5 Figs. 63A and 63B are diagrams illustrating the 

external appearance of the printer system according to 
the twenty-second embodiment; 

Fig. 64 is a flowchart showing the processing 
performed for a twenty- third embodiment; 
10 Fig. 65 is a diagram showing the functional 

arrangement of a system according to the twenty-third 
embodiment; 

Fig. 66 is a diagram showing a system for 
performing the overall process according to the twenty- 
15 third embodiment and input /output types relative to an 
external apparatus ; 

Fig. 67 is a diagram showing an example where a 
schedule is extracted from electronic mail; 

Fig. 68 is a flowchart showing specific processing 
20 performed for the twenty- third embodiment; 

Fig. 69 is a diagram showing an example schedule 
conflict; 

Fig. 70 is a diagram showing an example where a 
user aurally queries a system; 
25 Fig. 71 is a flowchart showing the processing for 

a twenty- fourth embodiment; 

Fig. 72 is a diagram showing an example 
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rearrangement of conflicting schedules; 

Fig. 73 is a flowchart showing the processing for 
re-planning conflicting schedules; 

Fig. 74 is a flowchart showing the processing for 
5 re-planning responses; 

Fig. 75 is a diagram showing an overall image 
including the input/output of a system according to a 
twenty-sixth embodiment ; 

Fig. 76 is a flowchart showing the overall 
10 processing performed by the system according to the 
twenty- sixth embodiment; 

Fig. 77 is a flowchart showing the processing 
performed by an input management unit; 

Fig. 78 is a flowchart showing the processing 
15 performed by a core unit; 

Fig. 79 is a flowchart showing the processing 
performed by an output management unit; 

Fig. 80 is a flowchart showing data derivation 
processing; 

20 Fig. 81 is a diagram showing an example document 

from which data are to be extracted; 

Figs. 82A and 82B are diagrams for explaining a 
specific block reading process; 

Fig. 83 which is composed of Figs. 83A and 83B are 
25 flowcharts showing determination procedures in a 
process for referring to date data; 

Fig. 84 is a diagram showing transmission of 



- 18 - 



information according to a twenty- seventh embodiment; 

Fig. 85 is a diagram showing transmission of 
information according to a twenty-eighth embodiment; 

Fig. 86 is a diagram showing transmission of 
5 information according to the twenty-eighth embodiment; 

Fig. 87 is a flowchart showing the processing for 
the twenty-eighth embodiment; 

Fig. 88 is a table showing knowledge of a common 
knowledge base; 

10 Fig. 89 is a table showing knowledge of a common 

knowledge base in a field specified on a cover page; 

Fig. 90 is a diagram showing an example where 
voice is used to refer to processing history to perform 
a process; 

15 Fig. 91 is a flowchart showing the processing 

performed according to a thirtieth embodiment; 

Fig. 92 is a diagram illustrating an example input 
document ; 

Fig. 93 is a diagram showing the contents of a 
20 conversation between a system and a user; 

Fig. 94 is a diagram showing an output document 
that a system prepares based on the judgement of the 
conversation ; 

Fig. 95 is a flowchart for an example where a 
25 required process is automatically performed after a 
user is queried; 

Fig. 9 6 is a diagram showing an example wherein a 
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system is operated by a controller; 

Fig. 97 is a diagram showing an example status 
monitor; 

Fig. 98 is a diagram showing an example wherein a 
5 controller recognizes a target model and displays a 
corresponding UI; 

Fig. 99 is a flowchart showing the processing 
performed by a controller for the example wherein the 
controller recognizes a target model and displays a 
10 corresponding UI; 

Fig. 100 is a flowchart showing the processing 
performed by a target model for the example wherein the 
controller recognizes a target model and displays a 
corresponding UI ; 
15 Fig. 101 is a diagram showing an example wherein a 

controller receives a UI from a target model and 
displays it; 

Fig. 102 is a flowchart showing processing 
performed by a controller for the example wherein the 
20 controller receives a UI from a target model and 
displays it; 

Fig. 103 is a flowchart showing processing 
performed by a target model for the example wherein the 
controller receives a UI from the target model and 
25 displays it; 

Fig. 104 is a diagram showing an example wherein a 
controller receives a UI from a target model via a 




wireless LAN rather than directly, and displays it; 

Fig. 105 is a flowchart showing the processing 
performed by a controller for the example wherein the 
controller receives a UI via a wireless LAN and 
5 displays it; 

Fig. 106 is a flowchart showing the processing 
performed by a target model for the example wherein the 
controller receives a UI via a wireless LAN and 
displays it; 

Iq 10 Fig. 107 is a flowchart showing the processing 

■- pi 

Jg performed by a server for the example wherein the 

rim 

IS controller receives a UI via a wireless LAN and 

r; displays it; 

r_ Fig. 108 is a diagram showing an example wherein 

Ir; 15 via a wireless LAN a controller receives a different UI 

4; for each user and displays it; 

^4 Fig. 109 is a flowchart showing the processing 

performed by a controller for the example wherein via a 
wireless LAN a controller receives a different UI for 

20 each user and displays it; 

Fig. 110 is a flowchart showing the processing 
performed by a target model for the example wherein via 
a wireless LAN a controller receives a different UI for 
each user and displays it; 

25 Fig. Ill is a flowchart showing the processing 

performed by a server for the example wherein via a 
wireless LAN a controller receives a different UI for 
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each user and displays it; 

Fig. 112 is a table showing a UI determined in 
consonance with a target model and user identification 
information; 

5 Fig. 113 is a diagram illustrating an example 

wherein a model in front that is to be operated 
performs a function that the model does not have; 

Fig. 114 is a diagram illustrating an example 
wherein a model in front that is to be operated 
10 performs a function that the model does not have; 

Fig. 115 is a diagram showing an example performed 
when communication can not be performed directly with a 
model in front that is to be operated; 

Fig. 116 is a flowchart showing the processing 
15 performed by a controller according to a thirty-ninth 
embodiment ; 

Fig. 117 is a diagram showing an example wherein 
an operation/action by a user and a time are jointly 
stored as history along with important associated 
20 information; 

Fig. 118 is a flowchart for the processing 
performed by a controller according to a fortieth 
embodiment ; 

Fig. 119 is a flowchart for the processing 
25 performed by a target model according to the fortieth 
embodiment ; 

Fig. 120 is a flowchart for the processing 
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performed by a server according to the fortieth 
embodiment ; 

Fig. 121 is a table showing example updated 
history information; 

Fig. 122 is a diagram illustrating an example 
wherein an operation is performed based on history; 

Fig. 123 is a diagram illustrating the example 
wherein the operation is performed based on history; 

Fig. 124 is a diagram showing an example structure 
for a controller; 

Fig. 125 is a diagram showing another example 
structure for a controller; 

Fig. 126 is a diagram showing an additional 
example structure for a controller; 

Fig. 127 is a diagram showing a functional 
arrangement according to a forty-second embodiment; 

Fig. 128 is a flowchart showing the processing for 
the forty-second embodiment; 

Figs. 129A, 129B, 129C, 129D, 129E and 129F are 
example task tables; 

Fig. 130 is a diagram showing an example 
input/output apparatus that can operate another 
apparatus ; 

Fig. 131 is a diagram showing an example 
input/output apparatus that can operate a system 
consisting of a plurality of apparatuses; 

Fig. 132 is a diagram illustrating example 
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input/output apparatuses -that can communicate with each 
other; 

Fig. 133 is a diagram illustrating an example 
input/output apparatus that can acquire information 
5 from another apparatus; 

Fig. 134 is a diagram illustrating an example 
input/output apparatus that can store information in 
another apparatus ; 

Fig. 135 is a diagram illustrating an example 

Q 

m 10 input /output apparatus that can send an instruction to 

another apparatus ; 
n Fig. 136 which is composed of Figs. 136A and 136B 

\~\ are flowcharts showing the processing for the task 

< COMMUNICATEE 

S H! 15 Fig. 137 which is composed of Figs. 137A and 137B 

H are flowcharts showing the processing for the task 

: 4 <PULL>; 

Fig. 138 which is composed of Figs. 138A and 138B 
are flowcharts showing the processing for the task 
20 <PUSH>; 

Fig. 139 which is composed of Figs. 139A and 139B 
are flowcharts showing the processing for the task 
< INDICATE > ; 

Fig. 140 which is composed of Figs. 140A and 140B 
25 are flowcharts showing the processing for the task 
<RESPOND> ; 

Fig. 141 is a flowchart showing the processing for 
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a response task relative to an information request type 
request; 

Fig. 142 is a flowchart showing the processing for 
a response task relative to an information storage type 
request; 

Fig. 143 is a flowchart showing the processing for 
a response task relative to an instructions type 
request; 

Fig. 144 is a diagram illustrating an example pull 
session; 

Fig. 145 is a diagram illustrating another example 
pull session; 

Fig. 146 is a diagram illustrating an additional 
example pull session; 

Fig. 147 is a diagram illustrating an example Hear 
On Demand session; 

Fig. 148 is a diagram illustrating a further 
example pull session; 

Fig. 149 is a diagram illustrating another example 
pull task; 

Fig. 150 is a diagram illustrating an additional 
example pull task; 

Fig. 151 is a diagram illustrating a session for a 
further example pull task; 

Fig. 152 is a diagram illustrating an example push 

task; 

Fig. 153 is a diagram illustrating an example push 



session; 

Fig. 154 is a diagram illustrating an example mail 
transmission; 

Fig. 155 is a diagram illustrating example 
instructions task INDICATE session; 

Fig. 156 is a diagram illustrating another example 
instructions task INDICATE session; 

Fig. 157 is a diagram illustrating additional 
example instructions task INDICATE session; 

Fig. 158 is a diagram showing the flow of an 
operation following the operational procedures 
performed for another apparatus; 

Fig. 159 is a diagram illustrating an example 
wherein a PC transmits a printing instruction to a 
printer; 

Fig. 160 is a diagram for explaining an example 
wherein a PC is employed as a server terminal; 

Fig. 161 is a diagram for explaining an example 
wherein a scanner and a printer are controlled by a PC; 

Fig. 162 is a diagram showing a relationship 
between a print job status change and a process 
associated with each status; 

Fig. 163 which is composed of Figs. 163A and 163B 
are flowcharts showing printing processing; 

Fig. 164 is a diagram showing a relationship 
between a print job and its processing, and a user's 
operation; and 
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Fig- 165 is a diagram showing an example display 
screen for a user for operating a print job, 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
5 The preferred embodiments of the present invention 

will now be described while referring to the 

accompanying drawings . 

[First Embodiment] 

Fig. 1 is a diagram illustrating a system 
10 arrangement according to a first embodiment in which a 

PC (Personal Computer) 101, a scanner 102, a color 

printer 103 and a monochrome printer 104 are connected 

to a network. 

In this embodiment, when an apparatus in a system 
15 wherein a plurality of apparatuses are connected 

together is instructed to execute a specific job, the 

apparatus analyzes the object of the job by examining 

its contents, and thus ascertains which job to execute. 

As a result of the analysis, the apparatus acquires 
20 necessary information concerning the job and thereafter 

executes it. 

If for the execution of a job there is available a 
more effective method or another and better apparatus 
than an instructed method or an assigned apparatus, an 
25 instruction is issued to the better apparatus or a 

notice is transmitted to a user to propose the use of 
the more effective method. Further, as a consequence 
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of "the examination of the contents of the job the 
instruction may be rejected. 

Job types are an operation input by a user, an 
execution of an instruction received from another 
5 apparatus, the acquisition of results by analyzing a 
job, or a job that is newly generated by an owned 
apparatus during an idling period. When, for example, 
a document is to be read by a scanner and printed by a 
printer, the following jobs are generated. 
m 10 ( 1 ) Analysis of the contents of an instruction that is 

,f% transmitted from a user to a scanner and that is 

f fs 

related to a document set in the scanner; 
n (2) A reading and analyzation process for a document, 

j" a and an instruction process for instructing a printer to 

15 print a document that has been read, both of which are 

ry 

generated by analyzing an instruction from a user that 
is received by the scanner (and the fact that the 
document has been set in the scanner ) . 

(3) Generation by the scanner of a printing 

20 instruction for a printer during the instruction 
process. 

(4) Generation of a printing process by the printer as 
a result of an analysis of the printing instruction 
received from the scanner . 

25 A method for generating and managing these jobs 

will later be described in detail by referring to an 
eleventh and following embodiments. 
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The procedures according to which a plurality of 
apparatuses execute the above described processes will 
be specifically described while referring to the 
drawings. 

5 Fig. 2 is a diagram illustrating the functional 

arrangement for performing the processing in this 
embodiment. A job reception unit 201 receives a job 
from a user or another apparatus. The received job is 
registered in a job table. A job analysis unit 203 

10 reads and analyses a job registered in the job table 
202. Another apparatus information acquisition unit 
204 acquires the attributes and the current states of 
the other apparatuses that are connected. A state 
determination unit 207 determines the state from the 

15 current states of the other apparatuses. 

An optimum planning unit 206 makes an optimal plan 
for the execution of a job. A job execution 
determination unit 207 determines whether a job should 
be executed, or when the job is to be executed, and 

20 determines whether an owned apparatus or another 

apparatus will execute the job. When another apparatus 
executes the job, another apparatus instruction unit 
208 sends an instruction to another apparatus for the 
job execution. When the owned apparatus executes the 

25 job, a job execution unit 209 executes the job. An 
execution notification unit 210 notifies a user that 
the execution instruction has been issued to another 
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apparatus . 

Fig. 3 is a diagram showing -the hardware 
arrangement: of the units that perform the processing in 
this embodiment. 
5 An I/O 301 exchanges input /output data with 

external apparatuses. A CPU 302 executes a program and 
controls the individual units. A ROM 303 is used to 
store programs, which correspond to individual 
flowcharts that will be described later and which are 

10 to be executed by the CPU 302, and fixed data. A RAM 
304 is used to temporarily store variables and 
intermediate data, such as the attributes and the 
current states of the other apparatuses acquired by the 
other apparatus information acquisition unit 204, that 

15 are generated during the processing. A program from an 
external source may be loaded into the RAM 304 and 
stored therein. 

Fig. 4 is a flowchart showing the processing 
performed for the first embodiment. 

20 At step S109, the contents of the job table are 

initialized. At step SI 10, in order to determine 
whether a job has been input, a check is performed to 
determine whether or not a job has been input by a 
user, a job has been input by a device that can be 

25 detected by the system, from results obtained by 

analyzing the job, or a new job has been generated by 
an owned apparatus during idling. When a job has been 
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input, at step Sill, the input job is added to the job 
table. At step S112, the job table is examined to 
determine whether there is job. If there is a job, at 
step S113 the job is extracted. At step S114 a 
5 received instruction is analyzed, and from the 

information thus obtained, the object of the job is 
acquired. At step S115, the analyzed instruction is 
employed to search for a job that is to be executed. 
At step S116 a job that is searched for at S115 is 

10 registered. 

At step SI 17 a check is performed to determine 
whether or not there is a job that has not yet been 
executed. If there is such a job, program control 
advances to step SI 18. At step SI 18 a check is 

15 performed to determine whether or not the job can be 

executed. When it is ascertained that the job can not 
be executed, program control returns to step SI 15 and a 
job that must be executed is searched for. At step 
SI 19 the job is executed and program control returns 

20 thereafter to step SI 17. 

If a user sends an instruction, for example, to 
read a document using a scanner and to print the 
document using a printer, it is assumed that, as a 
result of the analysis, the acquisition of a printed 

25 document is the object of a user. In this case, a 
document reading and analysis process, which is 
generated by analyzing information received by the 



scanner from the user, and a process for instructing a 
printer to print the document that has been read are 
searched for as jobs. These processes will be 
described in detail during the course of the eleventh 
and the following embodiments while referring to 
specific examples . 

Fig. 5 is a diagram showing a process for 
selecting from among a plurality of printers an optimal 
printer for performing printing. 

In Fig. 5, job information is input /transmitted as 
a command from a host computer or an instruction from a 
remote controller, or is obtained by analyzing a voice 
command issued by a user using natural language, or 
from bar code or a sentence read from a cover page. 

A device that has received the job information 
analyzes the job, and then selects a printer that can 
execute a process suitable for the job and outputs the 
data to the printer. Further, when there is an 
instruction for notification, the device selects an 
optimal medium and dispatches a notice thereto. 

Therefore, upon the receipt of a request from a 
user, the structure of the printer is referred to, and 
an appropriate printer is selected to perform the 
printing. At this time, in consonance with the 
situation, the processing can be allocated for a 
plurality of printers. Further, when there is an 
instruction for notification, an optimal medium is 




selected to dispatch a notification that the printing 
has been terminated along with the name of the printer 
that has performed the printing. 

A request submitted by a user concerns paper 
5 sizes, printing quality, colors and time limit (e.g., 
completion of printing by 5 o'clock). The printer 
configuration covers the printer types and the number 
of printers in the system, and their performances or 
their current statuses (whether the printers are 

10 normally operated, whether there is a printing queue, 
or whether paper or toner is insufficient). As for a 
medium and a method used for notification, a 
notification is transmitted by voice through a user's 
terminal or by telephone, or a notification message is 

15 transmitted to a pager, or a document describing the 

contents of a notification is transmitted by electronic 
mail or by facsimile. 

A situation where the individual units perform the 
processing shown in Fig. 5 will now be described while 

20 referring to the flowchart in Fig. 4. 

First, if, at step SI 10, an urgent instruction for 
outputting a specific document by five o'clock is 
transmitted to a host computer, at step Sill a job is 
added to a job table. At step S113, the job is 

25 extracted from the job table and at step SI 14 the 

received instruction is analyzed to obtain the object 
of the job. 
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At: step S115, "the following jobs are searched for 
following the analysis of the instruction. 

(1) Acquisition of specifications (paper size, 
printing quality, color document, etc. ) to be used for 

5 selecting a suitable printer for outputting a document, 
and of other information, such as time limit. 

(2) Selection from among the connected printers of a 
suitable printer for specifications ( 1 ) . 

( 3 ) Examination of the selected printer to determine 
10 whether or not any obstacle to printing output exists. 

(4) Output data to a selected printer. 

(5) Issuance of the selected printer and a 
notification that the data has been output. 

At step S116, the job found at step S115 is 
15 registered. The above process is repeated until, at 
step S117, there are no jobs to be searched for. At 
step S118 a check is instituted to determine whether or 
not the job can be performed. When the performance of 
the job is not possible, another job that must be 
20 executed is searched for. At S119, the jobs that have 
been searched for are sequentially performed. 
[ Second Embodiment ] 

Fig. 6 is a diagram for explaining an example 
wherein a remote controller (hereinafter referred to 
25 merely as a controller) is employed as a means for 
sending instructions to various devices. 
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A controller 605, which has a display, displays a 
control panel for, or information concerning a device, 
such as a printer 601 or 602, a facsimile machine 603 
or a PC 604, only by approaching the device, and sends 
5 instructions to the device to control it. Further, the 
controller 605 can control a remote device across a 
network via the device that is near the controller 605. 

The controller 605 can also monitor and display 
the status of each device, and can acquire the status 
10 of a remote device on the network via a device that is 
near the controller 605 . 
[ Third Embodiment ] 

Fig. 7 is a diagram showing an example where a 
document is read by a scanner and is transmitted to a 
15 specific printer. 

When a document has been read by a scanner 701, a 
user designates an addressee by voice, etc. , or a 
destination transmission is designated on a cover 
sheet, and is thus automatically determined. When a 
20 monochrome printer 702 is determined as an addressee, 

( 1 ) if the addressee is not in the ready for printing 
state, the document data is transmitted to another 
printer (e.g., a monochrome printer 703) for printing; 

(2) if a printer designated as an addressee can not 

25 print the document because of the conditions specified, 
such as color and paper size, data for only for the 
pertinent page is transmitted to another printer (e.g., 
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a color printer 704 if color printing is requested); 

(3) data for a document having a large number of pages 
are allocated to a plurality of printers to print; or 

(4) if a transfer destination is designated at the 
5 addressee, the data is transferred to the transfer 

destination . 

[ Fourth Embodiment ] 

Fig. 8 is a diagram showing an example where a 
document is read by a scanner and is transmitted to a 

10 specific printer that the scanner can not manage 
because it is in a different network domain, etc. 

When, a document is read by a scanner 701 in 
Austin, a user designates an addressee by voice, etc. , 
or an addressee that is described using a character 

15 string or a bar code on a cover sheet is read and 

recognized/ interpreted to determine the addressee- As 
a result, a monochrome printer 702 in Tokyo is 
determined to be the addressee. After the document 
data are transmitted to the monochrome printer 702: 

20 ( 1 ) when the monochrome printer 702 is not ready for 

printing, it transmits the data to another printer (a 
monochrome printer 703) for printing; 

(2) if the monochrome printer 702 is not ready for 
printing the received document data because of 
25 specified conditions, such as color and paper size, it 
transmits either all the data for the received document 
or only data for pertinent pages to another printer 
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(e.g., a color printer 704, if color printing is 
requested ) ; 

(3) if a document has a large number of pages, the 
monochrome printer 702 prints part of the data and also 

5 allocates printing data to a plurality of other 
printers; or 

(4) if a transfer destination is designated at the 
monochrome printer 702, the printer 702 transfers the 
data to the transfer destination. 

10 A difference between the third and the fourth 

embodiments is that in the third embodiment the scanner 
701 determines the addressee, and in the fourth 
embodiment the printer 702 that received the document 
from the scanner 701 determines the transfer 

15 destination. 

[Fifth Embodiment] 

Fig. 9 is a diagram showing an example wherein a 
document is read by a scanner and is transmitted to a 
specific printer that the scanner can not manage 

20 because it is in a different network domain, etc. 

When, a document is read by a scanner 701 in 
Austin, a user designates an addressee by voice or 
using a keyboard, or an addressee that is described 
using a character string or a bar code on a cover sheet 

25 is read and recognized/interpreted to determine the 
addressee. As a result, a monochrome printer 702 in 
Tokyo is determined as the addressee. After document 
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data are transmitted to the monochrome printer 702, 
they are further transferred to a personal computer 705 
that can perform a higher level operation. 

As a result, the personal computer 705 determines, 
5 from the entry "urgent" on the cover letter, for 

example, that the document is an urgent document. The 
personal computer 705 examines schedule data to find 
the location of the recipient of the document, and 
transfers the data to that location, for example, 

10 Atsugi. At this time, if the transfer destination is 
not a printer but is a different medium, such as a 
facsimile machine 706, medium conversion is performed 
before transmission. 
[ Sixth Embodiment ] 

15 Fig. 10 is a diagram showing an example wherein a 

document is read by a scanner and is transmitted to a 
specific printer that the scanner can not manage 
because it is in a different network domain, etc. 
When, a document is read by a scanner 701 in 

20 Austin, a user designates an addressee by voice, etc. , 
or an addressee that is described using a character 
string or a bar code on a cover sheet is read and 
recognized/interpreted to determine the addressee. As 
a result, a monochrome printer 702 in Tokyo is 

25 determined as the addressee. After document data are 
transmitted to the monochrome printer 702, they are 
further transferred to a personal computer 705 that can 
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perforin a higher level operation. As a result, the 
personal computer 705 identifies the recipient, 
prepares a notification sentence, and notifies the 
recipient of the arrival of the document. 
5 [Seventh Embodiment] 

Fig. 11 is a diagram showing an example wherein a 
notification concerning the status of a printer is 
transmitted. 

When a PC 1102 acquires the status of a printer 

10 1103 across a network and detects a change in the 

status, such as the occurrence of an error or an event, 
the PC 1102 determines a notification destination, such 
as a user or a manager, in consonance with the contents 
of the change, and employs a medium corresponding to 

15 the determined destination to transmit a notification. 
For example, to transmit a notification to a terminal 
1101 of a user or a manager, the PC 1102 uses 
electronic mail, or a telephone 1104, or transmits a 
message to a pager 1105- 

20 Therefore, in the example in Fig. 10, for example, 

a notification of the reception of the document may be 
issued using electronic mail or a pager, instead of the 
telephone 707. 
[ Eighth Embodiment ] 

25 In this embodiment, before executing an instructed 

job an apparatus notifies a user of the contents of the 
job in accordance with the type or the condition of the 
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job, or cancels a job that is determined to be 
unnecessary for a user. 

Fig. 12 is a flowchart showing the processing in 
this embodiment. Fig. 13 is a table showing, in 
5 consonance with the types and conditions of jobs, 
whether or not an advance notice should be issued. 

In Fig. 12, first, at step S120 a check is 
performed to determine whether or not a job constitutes 
necessary information for a user. If this is true, the 
£g 10 processes at step S122 and the following steps are 

performed. If not, at step S127 the job is canceled. 
At step S122, information concerning the advance notice 
y\ that corresponds to the job type and conditions in Fig. 

-;. 13 is referred to, or the contents described on a cover 

fit 

^ 15 sheet are analyzed to determine whether or not a notice 

^ for the process to be executed and the contents of the 

'"i process should be issued to a user prior to the 

performance of the instructed job. When advance notice 
is not required, program control moves to step S126. 
20 When advance notice is issued, program control moves to 
step S124, whereat a user is notified of the process to 
be performed and its contents. At step S125, the 
permission for the performance is received from a user, 
if necessary. At step S126 the job is performed. 
25 An explanation will be given for an example 

wherein, when the contents of a job call for the filing 
of information, an analysis of the system contents is 
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performed prior to the filing of information, and 
information concerning the contents is transmitted to a 
user. 

Fig. 14A is a diagram showing an example wherein 
5 filing process information to be filed is transmitted 
to a user in advance. A file storage unit 1401, a 
scanner 1402 and a user terminal 1403 are provided in 
this system. The processing performed by the scanner 
1402 in Fig. 14A will now be described while referring 

10 to the flowchart in Fig. 12. 

At step S120, since an instructed job is for 
information filing, it is assumed that this is 
necessary for a user. At step S122, while information 
designated in Fig. 13 is referred to, it is ascertained 

15 that a notice concerning information to be filed should 
be issued before the filing, and that the information 
should then be scanned. Program control then moves to 
step S124 to issue a notice concerning information to 
be filed. At step S124 the scanned information is 

20 transmitted to the user terminal 1403 via a network. 
At step S126, the filing job is performed. 

Next, an explanation will be given for an example 
wherein the information on the first page is received 
and analyzed, and when the information is not at all 

25 related to a user, the following information is not 

received . 

Fig. 14B is a diagram showing an example wherein 
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whether or not a large amount of data to be received is 
required by a user is determined by examining the first 
page, and the receipt of information for the following 
pages is canceled. A reception PC 1404 and a 
5 transmission PC 1405 are provided. 

The processing performed by the PC 1404 in Fig. 
14B will be explained while referring to the flowchart 
in Fig. 12. 

At step SI 20, the first page of the received data 

10 is analyzed. Since "For All Persons In Charge Of 

General Affairs Information" is entered on the first 
page, it is apparent that this document is not intended 
for a user who is not in charge of general affairs 
information, and the receipt of the information is 

15 determined to be unnecessary. Program control 

therefore moves to step S127. At step S127, the first 
page, which has been received, is canceled, and 
information for the second and the following pages is 
not received. The processing is thereafter terminated. 

20 [Ninth Embodiment] 

Fig. 15 is a flowchart showing the processing for 
a ninth embodiment. In this embodiment, an apparatus 
that is designated to perform the processing determines 
whether it or another apparatus should perform the 

25 processing, and as a result of the determination, 
either performs the processing itself or sends an 
instruction to another apparatus to perform the 
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processing , 

At step SI 50 a check is performed to determine 
whether or not a job has been input. If a job has been 
input, at step S151 the input job is entered in a job 
5 table. At step S152 the job table is examined to 

determine whether any jobs have been entered. If they 
have been, at step S153 a job is extracted from the 
table. At step SI 54 a received instruction is analyzed 
and the object of the job is apprehended using the 

10 result. At step S155 the status of the apparatus is 

examined. At step S156 the status of another apparatus 
consonant with the object of the job is examined. 

When, as the result of a comparison of the statues 
of the locally owned apparatus and other apparatuses, 

15 it is found that the locally owned apparatus is optimal 
for the performance of the job, program control moves 
from step S157 to step S158, whereat it is determined 
that the owned apparatus will perform the job, and at 
step SI 59 the job is performed by the locally owned 

20 apparatus. If there is an optimal apparatus other than 
the locally owned apparatus that has capabilities 
consonant with the object of the job, program control 
goes from step S160 to step S161, whereat it is 
determined that the other apparatus will perform the 

2 5 job, and at step SI 62 the locally owned apparatus sends 

a job execution instruction to the apparatus having 
capabilities consonant with the object of the job. At 
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step SI 63 the locally owned apparatus notifies a user 
that another apparatus has executed the job. 

If there is no apparatus having capabilities 
consonant with the object of the job, program control 
5 moves from step S160 to step S164, whereat a plan is 

prepared for the use of an optimal method that does not 
depart from the object of the job, and at step SI 65 the 
plan is proposed to a user. 

Fig. 16A is a diagram showing the transmission of 
lQ 10 information in this embodiment. 

ift When a user of a PC 101 provides for a printer 104 

□ a job for the output of information input by a scanner 

iVj 102, and when the printer 104 has, for example, a 

printing malfunction, the printer ascertains that it 
LfJ 15 can not perform the instructed job, and by 

•jT communicating with other apparatuses, it finds a 

printer 103 that can perform the instructed job. Thus, 
the printer 104 determines that the printer 103 can act 
for it and perform the job, and so instructs the 
20 printer 103. Further, the printer 104 transmits a 

notification to the PC 101 to inform the user of the 
action that it took. In Fig. 16A, the process flow for 
these activities is described by arrows. 

The processes performed by the individual units in 
25 Fig. 16A will be described while referring to the 
flowchart in Fig. 15. 

First, the process performed by the scanner 102 
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will be explained. At step S150 the scanner 102 scans 
a document and acquires a job, information concerning 
which is to be transmitted to the printer 104. At step 
S151 this job is entered in the job table, and at step 
S153 the job is extracted. At step S154, the scanner 
102 scans additional data and determines that the data 
is a job to be transmitted. At step S155, the scanner 
102 ascertains that it is operating normally and that 
no problem exists in the scanning and transmission of 
instructed information. 

At step S156 it is ascertained that the job can 
not be transmitted to another apparatus. At step S157, 
since optimally the scanner 102 should execute the job, 
program control moves to step S158. At step S158 the 
scanner 102 determines that it should scan the 
information, and at step S159 the scanner 102 scans the 
information and transmits it to the printer 104 via a 
route 1 on a network. 

At step SI 50, the printer 104 receives the job for 
the output of received information. At step SI 51, the 
printer 104 enters the job in the job table, and at 
step S153, it extracts it. At step S154, the printer 
104 ascertains from the information it receives that 
the information for the user from the PC 101 is to be 
printed on paper. At step S155 the printer 104 
determines that it is out of toner and disables 
printing. At step S116 the printer 104 searches, via 
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the network, for printers that can output the received 
information, and finds the printer 103 (route 2). 

Since, at step S157, the printer 104 can not 
execute the job, program control moves to step SI 60. 
5 Then, as at step S160 it is determined that the job for 
the output of information can be performed by the 
printer 103, at step S161 it is determined that the 
received information should be transmitted to the 
printer 103. At step S162 the printer 104 sends an 

10 instruction to the printer 103 to print the information 
that is to be received (route 3). At step S163 to 
notify the user that the instructed job was performed 
by the printer 103 electronic mail for the user is 
transmitted to the PC 101 (route 4). 

15 Fig. 16B is a diagram showing the transmission of 

information for a case where no apparatus having 
capabilities consonant with the object of a job is 
available . 

When the user of the PC 101 provides a job for the 
20 output to the printer 104 of information input by the 

scanner 102, and when the printer 104 disables printing 
because it is out of toner, the printer 104 
communicates with other apparatuses, finds the printer 
103, which is suitable for the object of the job, and 
25 determines its status. Then, since the printer 103 

disables printing because it is out of paper, and as 
there is no other printer available that is suitable 
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for the object of the job, a notice is issued to the PC 
101 to propose to the user an optimal plan whereby when 
either the printer 103 or 104 has recovered to the 
printing enabled state, that printer will perform the 
5 job. In Fig. 16B, the process flow for these actions 
is indicated by arrows. 

The procedures performed by the individual units 
in Fig. 16B will be explained while referring to the 
flowchart in Fig. 15. 
10 The process whereby the scanner 102 reads a 

document and transmits it to the printer 104 is 
performed in the same manner as was described 
previously. 

At step S150 the printer 104 receives a job for 
15 the output of received information. At step SI 51, the 
job is entered in the job table, and at step S153, it 
is extracted. At step S154, the printer 104 ascertains 
from the information it receives that the information 
for the user from the PC 101 is to be printed on paper. 
20 At step S155, the printer 10 ascertains that it is out 
of toner and printing is disabled. At step S156, the 
printer 104 communicates across the network with other 
printers to find one that can print the received 
information. The printer 104 determines that the 
25 printer 103 has also disabled printing because it is 
out of paper, and that no other appropriate apparatus 
is available. 
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Since, at; step S157, the printer 104 disables the 
performance of the job, program control moves to step 
S160. At step S160, as there is no apparatus that can 
perform the job for the output of information, program 
5 control goes to step S164. At step S164, planning is 
begun to prepare an optimal method for performing the 
printing, which is the object of the job for the output 
of information. As a result, it is determined that the 
optimal method involves the return of a printing 

10 apparatus to its normal operating status. At step 

S165, the proposed method, whereby either the printer 
104, which is out of toner, or the printer 103, which 
is out of paper, is to be recovered to the printing 
enabled state, is presented in a window shown in Fig. 

15 17 for approval by the user. 

Then, a response received from the user, or a 
change in the printer status is identified as an input 
job, and the same processing is again performed. In 
this fashion, the object of the user can be achieved. 

20 The above described planning will be described 

later in detail during the course of the explanation of 
specific examples for an eleventh and subsequent 
embodiments . 
[ Tenth Embodiment ] 

25 In the processing for the ninth embodiment, 

wherein, at steps S158 and S159, a locally owned 
apparatus determines to perform a job itself, and 
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performs the job, before another instructed job is 
performed, the possibility of the occurrence of a 
problem is analyzed in detail, as in the procedures in 
Fig. 17. When it is ascertained, in consonance with 
5 environmental conditions or the status of the job 

performance, that performing the job is not advisable, 
a notice concerning the problem may be issued, or the 
request for the performance of the job may be rejected. 
When, for example, an instruction for printing a 

10 secret document is issued and a user is not identified 
as an authorized user, the instruction is rejected. 

Fig. 18 is a flowchart showing the detailed 
procedures for the performance of a job. 

At step S180 a check is performed to determine 

15 whether or not a problem exists concerning the 

performance of the instructed job. At step SI 81 no 
problem affecting the performance of a job is found, at 
step S185 the job is performed. If a problem is found, 
at step SI 82 a check is performed to determine whether 

20 a notice concerning the problem that was discovered 

should be transmitted to a user, or whether the request 
for the performance of the job should be rejected. 
When it is ascertained that such a notice should be 
transmitted to a user, at step S184 the notice is 

25 transmitted to the user. When it is determined that 
the request for the performance of the job is to be 
rejected, at step S186 the request for the performance 
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of the job is rejected and a notice of rejection is 
transmitted to the user. 

An example wherein a user of the PC 101 issues an 
instruction to output to the printer 104 confidential 
5 information read by the scanner 102 will now be 

described while referring to Figs. 15 and 18. Fig. 19 
is a diagram showing example confidential information 
to be read. 

First, at step SI 50 the scanner 102 scans 

10 available information and receives a job for the 

transmission of information to the printer 104. At 
step S151 the job is entered in the job table, and at 
step SI 53, the job is extracted therefrom. At step 
S154 data are scanned, and it is ascertained the data 

15 constitute a job to be transmitted. At step S155, the 
scanner 102 determines that it is operating normally 
and that there is no problem affecting the scanning and 
transmission of the instructed information. 

At step S155 it is ascertained that the job can 

20 not be transmitted to another apparatus- Since, at 

step S157, it is ascertained that optimally the scanner 
102 should perform the job itself, program control 
moves to step S158, whereat the scanner 102 determines 
that it should scan the information itself. 

25 At step S180, as a result of the scanning of the 

information, it is determined to be confidential 
information whose output is inhibited. Since a problem 
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affecting the performance of the job exists, at step 
S182 it is determined that the job (the scanning and 
the outputting of information) should be rejected. At 
step S183 the job is rejected, and program control 
5 thereafter goes to step S186. At step S186, a notice 
is issued that the job was rejected. 

Similarly, when a plurality of users employ the 
same system, the current user can be identified by a 
log-in name, and an instruction for the output of the 

10 contents of files belonging to other users can be 

rejected. When, instead of an instruction from a user, 
a document is received from the outside and the 
destination user differs from the current user, the 
output of the received document and the notice of the 

15 receipt of the document are rejected, or the processing 
is halted until the destination user begins to employ 
the system. 
[ Eleventh Embodiment ] 

The processing for an eleventh embodiment will now 

20 be described while referring to Fig. 4. At step S109 
the job table is initialized. Then, at step S110 a 
check is performed to determine whether there is input 
from a user or from an apparatus that can be detected 
by the system. At step Sill an input analysis job is 

25 entered in the job table to analyze the contents input 
at step S110. At step S112 a check is performed to 
determine whether there is a job to be performed. If 
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there is such a job, it can be acquired at step S113. 
At step SI 14, the object comprising the background for 
the performance of the job is understood. 

At step SI 15 a possible process is planned by 
5 using the contents that are input, the status of a 
device that can be detected by the system, and the 
knowledge that the system has currently acquired. At 
step SI 16 a new job is added if necessary. At steps 
S117 to S119 the process as planned at step S115 is 

10 performed in consonance with the contents. The 

processing is repeated until no more jobs remain, and 
program control thereafter returns to step S110. 

A case where a user is to print <file A> will now 
be explained. Fig. 23 is a diagram showing the 

15 contents of <file A> , the colored picture of a car. 

Fig. 1 is a diagram showing the structure of the system 
in this embodiment. A user sends an instruction from a 
PC 101, and a monochrome printer 104 is set as a 
normally used printer. A color printer 103 is also 

20 set. 

At step S109 the job table is initialized- At 
step Sill a job that is to be performed when there is 
no input is added to the job table (Fig. 20). When 
file name <file A> is designated in a window in Fig. 21 
25 and printing for it is selected, at step SI 10 it is 
determined that there is input, and program control 
advances to step Sill. At step Sill, as is shown in 
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Fig. 22, an input analysis job is entered in the job 
table. Since, at step S112, there is a job to be 
executed, program control advances to step S113. At 
step S113 an analysis of the job to be performed (input 
5 "print <file A> 11 ) is acquired. At step S114 it is 

ascertained that the object of the input "print <file 
A>" is the printing of the <file A> . 

At step S115 planning is performed, based on the 
following conditions and statuses, for proposing, to a 
10 user that the printing be performed by a color printer. 

* <file A> includes a colored portion (the portion 
comprising the car in Fig. 23) 

* the color printer 103 that can be used is present 
(Fig. 1). 

15 At step S119 a query "Perform printing by a color 

printer?" is directed to a user (Fig. 24). At the same 
time, a job corresponding to an event is added to the 
job table when no response is received from the user 
(Fig. 25). Program control thereafter returns to step 

20 S110. 

When, at step SI 10, there is no input for ten 
minutes, at step S112 it is assumed that there is a job 
to be performed, and program control advances to step 
S113. At step S113, a job to be performed (when no 
25 response is received from a user) is acquired. 

At step SI 14 it is ascertained that the object for 
a job when there is no response from a user is to 
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determine whether a proposal can be accepted. At step 
SI 15 it is determined from the flow condition and the 
status that the user should be queried by telephone. 

* The telephone number of the current location of the 
user is acquired (schedule data in Fig. 26) 

* The printing priority is urgent (according to the 
schedule data in Fig. 26, file A is to be used at the 
business talk at 13:00). 

At step S119 a query, "Perform printing with a 
color printer? 11 is directed to the user (Fig. 27). At 
the same time, a job corresponding to an event is added 
to the job table when no response is received from the 
user (Fig. 28). Program control thereafter returns to 
step S110. 

When, at step S110, a user responds with "Yes, 
please" as is shown in Fig. 27, it is assumed that data 
input is to be performed, and program control moves to 
step Sill. At step Sill an input analysis job is added 
to the job table (Fig. 21). At step S112 it is assumed 
that there is a job to be performed, and program 
control moves to step SI 13. At step SI 13, a job to be 
performed, the input, "Yes, please" is analyzed. At 
step S114 it is ascertained that the object of the 
input, "Yes, please" is the acceptance of the proposal. 

At step S115 it is determined that <file A> is to 
be transmitted to and printed by the color printer 103. 
At step S118 <file A> is transmitted to and printed by 
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-the color printer 103. At the same time, a job for 
confirming whether or not the printing has been 
terminated normally is added to the job table (Fig. 
29). Program control thereafter returns to step S110. 
5 If, at step SI 10 there is no data input and at 

step S112, as is shown in Fig. 30, the printer status 
is changed from "Printing" to "Printed document in 
tray" it is assumed that there is a job to be executed, 
and program control advances to step S113. At step 

10 SI 13 a process corresponding to the job to be executed, 
printer status, is acquired. At step S114, from a 
printer status entry in Fig. 31, the status "Printed 
document in tray" means that there is a printed 
document in a printer tray, and the object of the job 

15 is understood to be the delivery of the printed 
document to a user. 

At step S115, a waiting state is specified because 
the user may soon come to pick up the printed document . 
At step S116, a job for confirming whether the user has 

20 come and has picked up the printed document is added to 
the job table (Fig. 32). Program control thereafter 
returns to step S110. If, at step S110, there is no 
input for ten minutes and at step SI 12 the printed 
document is still on tray, there is a job to be 

25 executed, and program control therefore advances to 

step S113. At step S113 a process corresponding to the 
job to be executed, when the printed document is still 
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on hand, is acquired. At step SI 14, from the table in 
Fig. 31 the status "Printed document on tray" means 
that a printed document is still in the printer tray, 
and it is assumed that the object is the delivery of 
5 the printed document to the user. Since at step S115 
the printed document has not been picked up for ten 
minutes, it is assumed that the user has forgotten to 
collect the document, and it is determined that a 
notice, "Printed document is in color printer tray" is 

10 to be issued (Fig. 33). Also, a job corresponding to 
the failure of the user to respond and a job 
corresponding to the collection of the printed document 
by the user are added to the job table (Fig. 34). 
Program control thereafter returns to step S110. 

15 When at step SI 10 there is no input and at step 

S112 the printer status is changed from "Printed 
document in tray" to "Normal" as is shown in Fig. 28, 
there is a job to be executed and program control 
advances to step S113. At step S113 a job 

20 corresponding to the job to be executed, printed 

document in tray, is re-evaluated. At step SI 14 the 
object for the rechecking of the job corresponding to 
the "Printed document in tray" is understood to be the 
deletion of an unnecessary job. At step S115, it is 

25 determined that the job of waiting for a response from 
the user should be deleted, and at step S119 the job of 
waiting for the user's response is deleted. A new job 
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is not added. While the job table is in its initial 
state, program control returns to step S110. 

Figs. 36 and 37 are tables showing plans, actions 
and rules . 

5 When a user presses the button "Print" in the 

printing window in Fig. 21, it is determined by 
referring to the plans and actions in Fig. 36 that the 
object of the user is the changing of the current state 
to the "Printed document in tray". In addition, since 
10 "Printed document in tray" is the assumption for the 
object "Acquire printed document" the final object of 
the user is determined to be "Acquire printed 
document" . 

When a user expresses agreement or disagreement by 
15 voice or by the manipulation of the mouse in response 
to the query in the inquiry window in Fig. 24 or the 
window in Fig. 33, or the telephone query in Fig. 27, 
it is determined that the object of the user is the 
statement of the agreement or disagreement relative to 
20 the query. 

In Fig. 37 the final object of the system is the 
thus constitutes the achievement of the object of the 
user. As the basis for this, the system must be 
stable. In addition, the system must (1) apprehend the 
25 object of the user and (2) must make a plan for the 
achievement of the object and execute it. 

To stabilize the system, not only an abnormal 
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timing for the system must be adjusted to the normal 
condition, but also the system must be powered off when 
it is not required in order to reduce costs and to 
prevent the system from becoming unstable. 
5 Fig. 38 is a diagram that for easier understanding 

shows only one part of the table in Fig. 37. 

In order to understand the object of the user, 
data that is input is identified. Further, 
corresponding to "acquire printed document" which is 

10 the object of the user in Fig. 36, "Transfer printed 
document to user 11 which is the object of the system. 

The presence of a printed document is the premise 
for the transfer of the printed document to a user, and 
the object is achieved by notifying the user of the 

15 location of the document. Of course, the user must 

collect the document or the document must be delivered 
to the user by some means in order for the user to 
acquire the document; in this embodiment, it is 
determined that when the printed document is no longer 

20 in the printer tray, that is the equivalent of the user 
having acquired the document. 

The presence of an original document and the 
printing conditions are required in order to obtain a 
printed document, and it is premised that a 

25 corresponding printing environment normally exists. In 
this situation, a corresponding printing environment is 
selected to perform printing. The original document 
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must be so prepared that its existence is constant. In 
addition, to clarify the printing conditions, only when 
there is a question concerning a condition need ask a 
user queries. When a printing environment is abnormal, 
5 the printing environment is normalized to maintain 

normal environment. When no response is received from 
a user within a predetermined period of time, 
additional planning must be performed to ascertain the 
intent of the user. 
Iq 10 The properties of each device are stored in a 

£3; 

memory in each device, or in the memory of the server 

m that manages each device. Fig. 39 is a table showing 

2 if 

iVg the properties of the devices. The statuses of the 

r devices are held in the devices, and are voluntarily 

Lrf 15 transmitted to other apparatuses by the devices, or by 

4f the passive transfer of them when requested by another 

* l *-4 server . 

In this system, the properties are inherent to the 
individual devices and do not change; they are acquired 
20 when the system is activated or when a connection 
across the network is altered. In the planning 
process, the statuses are acquired, as needed, or at a 
specific time interval, and are updated as information 
intended for the internal memory. 
25 [Twelfth Embodiment] 

An explanation will now be given for the 
processing performed in a case where, as in the 
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eleventh embodiment, a color printer is busy when <file 
A> is to be printed. 

First, when "Print <file A>" is input, it is 
entered in the job table, and its object is understood 
5 to be a request that printing be performed. Assume as 
the condition/situation that the colored portion is 
included in a document and a color printer is busy. A 
plan is devised to query a user concerning whether the 
job can wait until the color printer is not busy, or 
10 whether the data can be printed in monochrome. The 
^ query, "The color printer is busy. Wait, or print in 

S monochrome?" is presented to a user. When the user 

: . Z 

~ H ~ selects "Wait" the process is placed on standby until 

! i £ 

f the color printer is no longer busy. 

[U 15 [Thirteenth Embodiment] 

rtj 

=p When " <file A> was changed to <file A f >" is input, 

NJ it is ascertained that the updating of the job table is 

the object. As the condition/situation, the <file A> 
printing job is stored in the job table. Thus, a plan 
20 is made to query a user concerning the changing of the 
printing target to <file A f >. Then, the query "Print 
<file A f > instead of <file A> before amended?" is 
presented to the user. . 
[ Fourteenth Embodiment ] 
25 According to this embodiment, in a system wherein 

a plurality of apparatuses are connected together (in 
this case, they are connected across a network), when a 
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job for an individual apparatus is instructed, an 
analysis of the object of the job is performed by the 
appearance. When the apparatuses determine from their 
statuses and the status of the instructed apparatus, 
5 that the instructed apparatus should not execute the 
job, the performance of the job is transferred to 
another apparatus - 

Figs. 41A and 41B are diagrams in each of which is 
shown the condition where a plurality of apparatus are 

10 connected to a network. A network in Fig. 41A provides 
an environment wherein a scanner 411 and printers 412 
and 413 can determine the states of the other 
apparatuses. A network in Fig. 41B provides an 
environment wherein the scanner 411 and the printers 

15 412 and 413 can not determine the states of the other 
apparatuses . 

Fig. 42 is a diagram illustrating a functional 
arrangement for performing the processing in this 
embodiment. A job reception unit 421 receives a job 

20 from a user or from another apparatus. A received job 
is registered in a job table 422. A job analysis unit 
423 extracts the job from the job table 422 and 
analyzes it. Another apparatus information acquisition 
unit 424 determines the attributes and the current 

25 statuses of other connected apparatuses. A self state 
determination unit 427 determines the attribute and the 
current state of the self apparatus. 




An optimum planning unit 425 prepares an optimal 
plan for the performance of a job. A job performance 
determination unit 428 determines whether or not a job 
is to be performed, and whether the locally owned 
5 apparatus or another apparatus should perform the job. 
When the job is to be performed by the locally owned 
apparatus, a job performance unit 426 performs it. 
When another apparatus is to perform the job, another 
apparatus job transfer unit 429 transfers the job to 
'1% 10 another apparatus. A transfer notification unit 430 

>: notifies a user of the results obtained by the job 

performance or the transfer of the job to another 
*4 apparatus. Fig. 43 is a main flowchart showing the 

f processing performed for this embodiment. 

r 

FU 15 At step S430, in order to determine whether a job 

,,5 has been input, a check is performed to determine 

Sj whether or not a job has been input by a user, a job 

has been input by a device that can be detected by a 
system, based on the results obtained by analyzation of 
20 the jobs, or a new job that has been generated by the 
locally owned apparatus while idling. When a job has 
been input, at step S431, an input job is added to the 
job table. At step S432, the job table is examined to 
determine whether there is a job. If there is a job, 
25 at step S433 the job is extracted. At step S434 a 
received instruction is analyzed, and from the 
information attained by the analyzation, the object of 
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the job is obtained. At step S435 a check is performed 
to determine whether or not a job for another apparatus 
is present. If there is such a job, at step S436 
control of another apparatus is initiated. If there is 
5 no such job, at step S437 a control process the locally 
owned apparatus is initiated. 

Fig. 44 is a flowchart for the process for 
controlling another apparatus performed at step S436 in 
the main flowchart. 

10 At step S440 the status of an apparatus that is 

designated in the job is examined, and at step S441, 
based on the status, a check is performed to determine 
whether a problem affecting the performance of the job 
by the designated apparatus exists. If no such problem 

15 exists, at step S442 the job is transmitted to the 
designated apparatus. If a problem affecting the 
performance of the job by the designated apparatus 
exists, at step S443 a check is performed to determine 
whether or not there is an apparatus other than the 

20 designated apparatus that is suitable for the 

performance of the job. If there is such an apparatus, 
its status is examined. As a result, if there is 
another apparatus that can execute the job, at step 
S445 the job is transferred to that apparatus, and at 

25 step S446 the user is notified of that the job was 
transferred to that apparatus. When there is no 
apparatus other than the designated apparatus that can 
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perform the job, or when another suitable apparatus can 
not be used at that time, at step S447 an optimal plan 
is devised that will not degrade the object of the job, 
and at step S448, the plan is proposed to the user. 

Fig. 45 is a flowchart for the process for 
controlling the locally owned apparatus performed at 
step S437 in the main flowchart. 

At step S450 the status of the locally owned 
apparatus is examined, and at step S451, based on the 
status, a check is performed to determine whether a 
problem affecting the performance of the job by the 
locally owned apparatus exists. If no such problem 
exists, at step S452 the job is performed by the 
locally owned apparatus. If a problem affecting the 
performance of the job by the locally owned apparatus 
exits, at step S453 a check is performed to determine 
whether or not there is an apparatus other than the 
locally owned apparatus that is suitable for the 
performance of the job. If there is such an apparatus, 
its status is examined. As a result, if there is 
another apparatus that can execute the job, at step 
S455 the job is transferred to that apparatus, and at 
step S456 the user is notified that the job was 
transferred to that apparatus. When there is no 
apparatus other than the locally owned apparatus that 
can perform the job, or when another suitable apparatus 
can not be used at that time, at step S457 an optimal 
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plan is devised that will not degrade the object of the 
job, and at step S458, the plan is proposed to the 
user. 

Fig, 46 is a diagram showing the condition that 
5 exists when, while, in Fig. 41A, an instruction is 
issued to output to the printer 412 along route A 
information that has been read by the scanner 411, the 
other apparatus information acquisition unit 424 of the 
scanner 411 discovers a malfunction at the printer 412, 
10 and this information is output to the printer 413 along 
route B . 

Fig. 47 is a diagram showing the condition that 
exists when, while, in Fig. 41B, the scanner 411 is 
instructed to output the information that is read to 

15 the printer 412 along route A and to confirm the status 
of the printer 412, since the printer 412 is located in 
a remote area, the scanner 411 takes the status of the 
network into account and unconditionally transmits the 
information to the printer 412, which then, because a 

20 malfunction has occurred there, transfers the received 
information (job) to the printer 413. 

The processing performed in Figs. 46 and 47 will 
be described while referring to the flowcharts shown in 
Figs. 43 to 45. 

25 When in Fig. 46 a job is to be output by the 

scanner 411 and transferred to the printer 412, the 
scanner 411 performs the following process. 
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First, at step S430 the scanner 411 receives a job 
for the scanning of information and the transmission of 
it to the printer 412. At step S431 the job is entered 
in the job table, and at step S433, it is extracted 
5 therefrom. At step S434 the scanner 411 scans the data 
and ascertains the job involves the transmission of 
data to a designated printer. At step S435 it is 
determined that there is a job, which is to be executed 
by another apparatus, for the output of the scanned 

10 information by the printer 412. Based on this 

determination, at step S436 program control moves to 
another apparatus control process in Fig. 44. 

At step S440 the current state of the printer 412 
is acquired. Then, information that the printer 412 

15 has malfunctioned is received, and at step S441 it is 
determined that printing by the printer 412 is 
disabled. In this case, besides a malfunction, the 
printer 412 can also be in an out of paper or out of 
toner disabled state. At step S443, it is assumed that 

20 the designated printer 412 can not execute the job, and 
another apparatus that can execute the job is searched 
for by communicating with the other apparatuses across 
the network. At step S444 it is found that the printer 
413 can execute the job and the performance of the job 

25 is assigned to it. 

At step S445 the scanned information is 
transmitted to the printer 413. And at step S446, 
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since a process other than the one instructed is being 
performed, the user is notified of the job alteration. 
Since the process for the instructed job has been 
terminated, the system waits for the next job. 
5 Following this, a process in Fig. 47 where a job 

is to be output by the scanner 411 and transmitted to 
the printer 412 will now be explained. 

First, at step S430 the scanner 411 receives a job 
for the scanning of information and the transmission of 

10 the information to the printer 412. At step S431 the 
job is entered in the job table, and at step S433, it 
is extracted therefrom. At step S434 the scanner 411 
scans the data and ascertains the job is for the 
transmission of data to a designated printer. At step 

15 S435 it is determined that there is a job, which is to 
be executed by another apparatus, for the output of the 
scanned information by the printer 412. Based on this 
determination, at step S436 program control moves to 
another apparatus control process in Fig. 44. 

20 At step S440, while the scanner 411 communicates 

with the printer 412 and attempts to acquire its 
status, it is found that the printer 412 is in a remote 
area and the scanner 411 decides to transmit the 
information, regardless of the status of the printer 

25 412. At step S441 it is determined that there is no 
problem with the printer 412 since it is in a remote 
area, and at step S442 the scanned information is 
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•transmitted to the printer 412, which is the designated 
apparatus, and the job is transferred thereto. Since 
the instructed job has been terminated, the scanner 411 
waits for the next job. 
5 At step S430 the printer 412 receives the job for 

outputting the received information. At step S431 the 
job is entered in the job table, and at step S433, it 
is extracted therefrom. At step S434 the printer 412 
understands that the job is for the printer 412 to 

10 output the received information. At step S435 it is 

determined that there is no job for another apparatus, 
and at step S437 program control moves to an own 
apparatus control process in Fig. 45. 

At step S450 the current state of the printer 412 

15 is examined, and at step S451 the printer 412 judges by 
itself that the printer 412 has a malfunction and is in 
a printing disabled state. In this case, besides the 
malfunction, the printer 412 can be in such an output 
disabled state as out of paper or out of toner. At 

20 step S453, another apparatus that can execute the job 
is searched for by communication with the other 
apparatuses across the network. 

At step S454 the printer 413 that can execute the 
job is found, and the execution of the job at the 

25 printer 413 is determined. At step S455, the printer 
412 transmits the received information to the printer 
413. At step S456, since the process other than being 
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instructed is performed, the user is notified of the 
alteration of the job. In this embodiment, such a 
notice is issued to the instruction source user. Since 
the output destination is far away from the instruction 
5 source, the notice may be issued to a recipient. Since 
the instructed job has been terminated, the system 
waits for the next job. As at the printer 413 there is 
no processing problem and the printer 413 can perform 
the job for the output of the received information, the 

10 printer 413 performs the job. 

In the above embodiment, the individual units 
automatically transfer the job. If there is a problem 
at each apparatus, a user may provide a transmission 
order for the transfer of the job by the apparatuses. 

15 In this embodiment, a user inputs as a job the 

employment of a designated printer to print a scanned 
document; however, a second and a third printer may at 
the same time be set for the job, so that if the 
designated printer malfunctions they can replace it and 

20 perform the printing. Thus, at step S443 or S453, the 
second or the third printer can be selected as a 
suitable apparatus for the job and its status is 
examined. A second or a third printer may be set as a 
printer to receive a job and to replace a designated 

25 printer when it is in a printing disabled state. 

However, even when a second or the third printer is not 
set for the job, the printer that receives the job can 




transfer it: in consonance with destinations that are 
set internally. 

In addition to a malfunction and an operation 
disabled state, the following conditions can cause the 
5 performance of a job to be canceled: the 

specifications may be inappropriate for the performance 
of the job, e.g., a case where a job for the printing 
of color data is issued to a monochrome printer; or 
immediate processing of the job may not be possible 

m 10 because there are too many jobs in a queue. 

[Fifteenth Embodiment] 

?5 In this embodiment, in a process for determining 

5.5 

[75 the status of the locally owned apparatus at step S450 

in Fig. 45, schedule information for a user, which is 
Lfj 15 the object, is examined as a factor that is used to 

^ determine whether a problem exists that will affect the 

iLJ 

N performance of a job involving the output to a user of 

urgent information received by a printer. 

An explanation will be given for a process that is 
20 to be performed in the environment shown in Fig. 48, 
where a user, to whom urgent information is to be 
transmitted, is currently at an outside location at 
which a printer 412 is available that is connected to a 
network . 

25 In the procedures in Fig. 43, at step S434 it is 

ascertained that the object of the job is the 
transmission to a user of urgent information that has 
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been received. At: step S435, since at this time there 
is no job for another apparatus, program control moves 
to step S437. At step S437 the process advances to the 
locally owned apparatus control process in Fig. 45. 
5 At step S450, in order to determine the status of 

the locally owned apparatus, the schedule of a user who 
is designated as a transmission source is confirmed by 
the PC 414, and it is ascertained that the user is at 
an outside location. Since the user is at an outside 

10 location, at step S451 it is determined that a problem 
related to making urgent contact exists at the locally 
owned apparatus. At step S453 a check is performed to 
determine whether or not a printer that can output 
information is present at the user's destination. At 

15 step S454 it is confirmed that there is a printer 412 
that can output information, and it is determined that 
the printer 412 will be used to output the information. 
At step S455 the information is transmitted to the 
printer 412. At step S456 the users at the 

20 transmission source and at the reception destination 
are notified that the job has been transferred. 

When, at step S443, there is no printer available 
to which to transfer the job but there is another 
apparatus that can be employed by the user, such as a 

25 PC, a facsimile machine or a telephone, at step S444 it 
may be determined to perform the job by using one of 
the available apparatuses. Since the information can 
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not be transmitted unchanged, at step S445 the 
information to be output to the printer is converted 
into an electronic mail document or a facsimile 
document, or is changed to voice information for the 
5 output. 

At step S446, depending on the medium used for the 
output, a notice that the job was transferred to 
another medium need not be issued, and can be 
transmitted by using the same medium. For example, 

£0 10 when a telephone is used to transmit the information, 

it is better to notify a user of the job transfer 

q before or after the transmission of the information. 

[75 Instead of the telephone, at step S443, the above 

r_= PC, the facsimile machine or the telephone may be 

SI ^ 

: » r 

LJ1 15 determined to be a suitable apparatus to replace the 

^ printer for the job. At step S447 the employment of 

^ such an apparatus may be determined to be an optimal 

plan that does not degrade the object of the job, and 
at step S448 the plan may be proposed to a user. 
20 [Sixteenth Embodiment] 

Fig. 49 is a flowchart showing the processing for 
this embodiment. Steps S498 to S500 are additionally 
provided for the flowchart in Fig. 43 as a process to 
be performed when there is no job. At step S498 the 
25 current operating statuses of other apparatuses are 
examined, and at step S499 a check is performed to 
determine whether or not a job that the locally owned 
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apparatus can perform is being held in another 
apparatus and not being performed. If such a job is 
being held in another apparatus, at step S500 the job 
is extracted from the pertinent apparatus and is 
5 entered in the job table for the locally owned 

apparatus, while it is deleted from the job table for 
the pertinent apparatus. In this embodiment, 
therefore, an apparatus finds a job by itself and 
processes it. 

10 Fig. 50 is a conceptual diagram showing the 

situation for this processing. Since no job is present 
in a printer 412, and a printer 413 has ten jobs in a 
queue, a printer 415 acquires a job from the printer 
413 and processes it. 

15 The process performed by the printer 415 is as 

follows. At step S492 there is no job, and program 
control moves to step S498. At step S498 the current 
statuses of the other printers, 412 and 413, are 
examined. At step S499 it is determined that at the 

20 printer 413 there is a job that the printer 415 can 
execute. At step S500 the printer 415 extracts that 
job from the printer 413, and enters it in its job 
table. Then, the extracted job is deleted from the job 
table for the printer 413. Since at step S490 there is 

25 no job input, and at step S492 there is a job that was 
entered previously, program control moves to step S493 
to execute the job. 




The operations for the performance of the job by 
the locally owned apparatus, which were explained in 
the fourteenth and the fifteenth embodiments, are 
performed, and the processing is thereafter terminated. 
5 [Seventeenth Embodiment] 

Fig. 51 is a diagram showing the transmission in 
this embodiment of a print job and other information in 
a system employing apparatuses. The heavy arrows are 
used to depict the transmission routes for the print 
m 10 jobs, and the broken line arrows are sued to depict the 

,g transmission routes for other information. 

PJ The print job in this embodiment includes 

Ltl 

I j parameters that are referred to during printing: the 

document to be printed; the number of print copies; the 
LH 15 print quality; and the print size. The other 

*tr information includes the statuses, such as printing, 

normal, or out of paper, of individual printers, and a 
schedule for a print job processed by a print job 
scheduling unit. 
20 In Fig. 51, in a client machine 510, a print job 

generation unit 512 generates a print job, and stores 
it in a print job memory unit 513. Then, the print job 
is transmitted by a print job transmission unit 514 in 
the client machine 510 to a server machine 511 for 
25 managing printers. 

In the server machine 511, a print job reception 
unit 516 receives the print job. When the parameters 
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of the print job have not yet been set, the print job 
is transmitted to a print job automatic setup unit 517, 
and when the parameters have already been set, the 
print job is transmitted to a print job automatic 
5 change unit 518. 

The print job automatic setup unit 517 sets print 
parameters while taking into consideration the urgency, 
economy and the quality that are acquired by the print 
job. The print job automatic change unit 518 can 

10 perform the same process for changing parameters that 
have previously been set. The print jobs that are set 
or altered are registered in a print job schedule 
included in a print job scheduling unit 520 and are 
sequentially read to a print job transmission unit 519, 

15 or the print job may be directly transmitted to the 

printer job transmission on the unit 519 from the print 
job automatic setup unit 517 or the print job automatic 
change unit 518. The printing is then performed. 

The status, such as in printing, normal or out of 

20 paper, of an available printer is acquired by a printer 
state acknowledgement unit 515, and is transmitted to 
the print job automatic setup unit 517 and the print 
job automatic change unit 518 where it is employed for 
the setup or change of the print job. Similarly, the 

25 printing schedule for the print job included in the 
print job scheduling unit 520 is also transmitted to 
these units 517 and 518 for their employment. 
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As is shown in Fig. 51, a print: job may be 
transmitted along a different route. When a machine 
for generating a print job is the same as a machine for 
managing printers, the print job may not be transmitted 
5 via the print job transmission unit 514. 

Fig. 52 is a flowchart showing the processing 
performed by the print job automatic setup unit 517. 
The print job automatic setup unit 517 automatically 
sets parameters within a range that satisfies a 

10 designated quality and the printing can be performed as 
rapidly and as economically as possible. 

In Fig. 52, at step S520 a factor for determining 
the printing time and the amount of consumed toner is 
initialized in preparing for the simulation. Then, at 

15 step S521 a quality determination variable is 

initialized to a maximum quality value (e.g., 5), and 
at step S522 the period of time for the printing is 
simulated. At step S523 a check is performed to 
determine whether the simulated period of time for the 

20 printing equals a specified value. When the time 

period for the printing equals a specified value, at 
step S524 the consumption of toner is simulated. At 
step S525 a check is performed to determine whether the 
simulated consumption of toner equals a specified 

25 value. When the consumption of toner equals a 

specified value, the job is changed to the current 
quality determination variable value at step S526. 
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When, at step S523 or S525, the period of time for 
the printing or the consumption of toner is not 
equivalent to its specified value, at step S527 the 
quality determination variable is decremented by one. 
5 At step S528 a check is performed to determine whether 
or not the resultant quality determination variable 
satisfies is equivalent to a designated quality. When 
the designated quality is satisfied, program control 
returns to step S522. When the designated quality is 

10 not satisfied, the processing is terminated with an 
automatic setup disabled state. 

Fig. 53 is a table showing the period of time for 
the printing and the consumption of toner for one 
character corresponding to different specified 

15 qualities for the printers A521 and B522. These values 
may be fixed values inherent to individual printers, or 
may be dynamic variable values, which are averages 
obtained for printing periods or for consumption of 
toner that were measured in the past . 

20 Fig. 54 is a diagram showing the contents of a 

print job that is to be output in this embodiment, and 
parameters that were designated by a user. In the 
system for the embodiment, the contents of a print job 
in Fig. 54 and the set-up parameters are collectively 

25 called a print job. In this embodiment, calculated in 
terms of characters, 500 characters are employed as the 
printing quantity for the print job in order to 
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simulate the period of time for the printing and the 
consumption of toner. In addition, it is assumed that 
for urgency a user will designate a period of only one 
minute, and will keep the default values for economy 
5 and quality. 

In the example shown in Fig. 54, when the print 
job automatic setup unit 517 is activated, at step S520 
in preparing of simulation, a factor for determining 
the period of time for the printing and the consumption 

10 pf. toner is initialized. As defined in the table in 

Fig. 53, for example, the periods of time required for 
the printing of one minimum quality ( = 1 ) character by 
the printer A521 and the printer B522 are, on average, 
100 mS and 200 mS respectively, and the consumption of 

15 toner for one character are 100 mg and 200 mg. 

Following this, at step S521 the quality 
determination variable is initialized to the maximum 
quality value (e.g., 5), and at step S522 the printing 
period of time for the printing is simulated. If, for 

20 example, the printers A521 and B522 are prepared for 

printing and no print job is scheduled in the print job 
scheduling unit 520, printing can be initiated 
immediately, and no waiting time is required. Thus, 
when the target print job includes 500 characters, as 

25 simulation, printer A takes 250 seconds to print the 
characters and printer B takes 500 seconds . 

Then, at step S523 it is determined that this 
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condition does not: satisfy the one minute urgency 
period that is specified by the parameter in Fig. 54, 
and at step S527 the quality determination variable 
value is decremented by one. 
5 Since the quality is not specified in the 

parameter in Fig. 54, program control returns to step 
S522 for simulation of the period of time for the 
printing. When, as the result of the repetition of the 
above process, the quality is set to the minimum 

10 quality of 1, the period of time for the printing by 
the printer A521 is 50 seconds, which satisfies the 
urgent figure. Therefore, the quality is set to the 
minimum quality determination variable value ( = 1 ) , and 
the processing is normally terminated. 

15 If the quality is set to 2 or higher, it does not 

meet the parameters set by the user, and the processing 
is terminated as an automatic setup disable error. 
[Eighteenth Embodiment] 

Fig. 55 is a flowchart showing the processing 

20 performed by a print job automatic change unit 518. 

The print job automatic change unit 518 automatically 
changes a parameter within a range that satisfies a 
specified urgency and economy so as to print data 
having as high a quality as possible. When all the 

25 specified parameters can not be satisfied, the most 
important designated parameter is satisfied and the 
others are changed as slightly as possible. 
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Fig. 56 is a diagram showing the coni:ents of a 
print job that is to be output in this embodiment, and 
parameters designated by a user. In the system of the 
embodiment, the contents of a print job in Fig. 56 and 
5 the set-up parameters are collectively called a print 
job. In this embodiment, calculated in terms of 
characters, 500 characters are employed as the printing 
quantity for the print job in order to simulate the 
period of time for the printing and the consumption of 

O 

gj 10 toner. In addition, it is assumed that a user 

designates all of the parameters and particularly 
iZ specifies that the quality is the most important, 

rl In the example shown in Fig. 55, when the print 

^ job automatic change unit 518 is activated, at step 

^ 15 S520 in preparing for the simulation, a factor for 

determining the period of time for the printing and the 
*""4 consumption of toner is initialized. As is defined in 

the table in Fig. 53, for example, the period of time 
required for the printing of one minimum quality ( = 1 ) 
20 character by the printer A521 and the printer B522 are, 
on average, 100 mS and 200 mS respectively, and the 
consumption of toner for one character are 100 mg and 
200 mg. 

Following this, at step S551 the quality 
25 determination variable is initialized to the maximum 

quality value (e.g., 5), and at step S552 the period of 
time for the printing is simulated. If, for example, 
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the printers A521 and B522 are prepared for printing 
and no print job is scheduled in the print job 
scheduling unit 520, printing can be initiated 
immediately, and no waiting time is required. Thus, 
5 when the target print job includes 500 characters, as 
simulation, printer A takes 250 seconds to print the 
characters and printer B takes 500 seconds. 

Then, at step S553 it is determined that this 
condition does not satisfy the one minute urgency 

10 period that is specified by the parameter in Fig. 56, 
and at step S556 the quality determination variable 
value is decremented by one. 

Since, at step S557, the quality in the parameter 
in Fig. 56 is not satisfied, program control returns to 

15 step S552 for simulation of the period of time for the 
printing. As a result of the repetition of the above 
process, even when the quality is the one (= 3) 
designated by the user, the urgency period can not be 
satisfied. When, at step S556, the quality 

20 determination variable value is decremented by one, at 
step S557 the quality specified by the user is still 
not satisfied, and program control therefore moves to 
step S558. At step S558 a check is performed to 
determine whether the quality is the most important 

25 parameter. When the quality is the most important 
parameter, at step S560 the print job is changed in 
accordance with the specified quality, and the 
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processing is normally terminated. 

For a case where the urgency is specified as the 
most important, when the quality is set to the minimum 
quality of 1, the printing period for the printer A521 
5 is 50 seconds, which conforms to the urgency setting. 
Therefore, the quality is set to the quality 
determination variable value ( = 1 ) , and the processing 
is normally terminated. 
[Nineteenth Embodiment] 

10 Fig. 57 is a diagram showing the transmission of a 

print job and other information in a system in this 
embodiment employing apparatuses . The thick arrows 
describe the transmission routes for print jobs and the 
broken line arrows describe the transmission routes for 

15 other information. 

The print job in this embodiment includes 
parameters that are referred to during printing : a 
document to be printed; the number of print copies; the 
print quality; and the print size. The other 

20 information includes statuses, such as printing, 

normal, or out of paper, of individual printers, and a 
schedule for a print job that is included in a print 
job scheduling unit 520. 

In Fig. 57, in a client machine 510, a print job 

25 generation unit 512 generates a print job and stores it 
in a print job memory unit 513. Then, the print job is 
transmitted by a print job transmission unit 514 in the 
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client machine 510 to a server machine 511 for managing 
printers . 

In the server machine 511, a print job reception 
unit 516 receives the print job and transmits it to a 
5 print job interpretation unit 571. The print job 

interpretation unit 571 interprets an instruction from 
a user that is included in the received print job, and 
converts the instruction into information, such as a 
parameter, that can be processed by the system. When 

10 the parameters of the print job have not yet been set, 
the print job is transmitted to a print job automatic 
setup unit 517, and when the parameters have already 
been set, the print job is transmitted to a print job 
automatic change unit 518. 

15 The print job automatic setup unit 517 sets print 

parameters while taking into consideration the urgency, 
economy and quality values that are acquired from the 
print job. The print job automatic change unit 518 can 
perform the same process to change the parameters that 

20 have been set. The print jobs that are set or altered 
are registered in a print job schedule included in a 
print job scheduling unit 520 and are sequentially read 
to a print job transmission unit 519, or the print job 
is directly transmitted to the print job transmission 

25 unit 519 from the print job automatic setup unit 517 or 
the print job automatic change unit 518. The printing 
is then performed. 
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The status, such as in printing, normal or out of 
paper, of an available printer is acquired by a printer 
state acknowledgement unit 515, and is transmitted to 
the print job automatic setup unit 517 and the print 
job automatic change unit 518 where it is employed for 
the setup or change of the print job. Similarly, the 
printing schedule for the print job included in the 
print job scheduling unit 520 is also transmitted to 
these units 517 and 518 for their employment. 

As is shown in Fig. 57, a print job may be 
transmitted along a different route. When a machine 
for generating a print job is the same as a machine for 
managing printers, the print job may not be transmitted 
via the print job transmission unit 514. 

In this embodiment, as is shown in Fig. 58A, the 
parameters of the print job are set ambiguously by 
using natural language. Fig. 58B is a diagram showing, 
relative to an input character string in the natural 
language, the meanings represented by character strings 
according to the parameters and the items required to 
complete the meanings. The print job interpretation 
unit 571 refers to the meaning corresponding to an 
input character string in the natural language in Fig. 
58B and the item required to complete the meaning, and 
interprets the instruction given in the natural 
language that is received as a print job setup 
parameter as follows. 
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From the character string "Print" in the input 
natural language, "printing" is the action that is the 
object of the user according to the meaning of the 
character string. Further, it is assumed that the 
5 required item "object" indicates the contents of a 
print job that was input at the same time, and that 
"quality" and "number of sheets" should be designated 
at a portion in the input natural language that has not 
yet been interpreted. Thus, the character string "one 
10 copy for client and five copies for members" is 

interpreted as an instruction to print one "sheet" 
having a high "quality" and five "sheets" having a 
desired "quality" . 

The processes hereinafter performed by the print 
15 job automatic setup unit 517 and the print job 

automatic change unit 518 are the same as those in the 
seventeenth and the eighteenth embodiments. 
[ Twentieth Embodiment ] 

In this embodiment, as is shown in Fig. 59, the 
20 print job parameters are set based on the distribution 
destination described in a document, which is the 
contents of the print job, and on the number of sheets. 
The action of the job is printing. 
Printing for a "client" is interpreted as high 
25 quality printing, and the number of sheets is one. 

"ABC project" is interpreted as a "member use" and 
the printing quality is arbitrary, the number of sheets 
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being five. A user acquires in advance information 

that he or she is a member of the ABC project. 

The processes hereinafter performed by the print 

job automatic setup unit 517 and the print job 
5 automatic change unit 518 are the same as those in the 

seventeenth and the eighteenth embodiments. 

[ Twenty- f i r st Embodiment ] 

Fig. 60 is a diagram showing the transmission of a 

print job and other information in a system in this 
10 embodiment employing apparatuses. The heavy arrows 

describe the transmission routes for print jobs and the 

broken line arrows describe the transmission routes for 

another information . 

The print job in this embodiment includes 
15 parameters that are referred to during printing: the 

document to be printed; the number of print copies; 

print quality; and print size. The other information 

includes statuses, such as printing, normal, or out of 

paper, of individual printers, and a schedule for a 
20 print job that is included in a print job scheduling 

unit 520. 

In Fig. 60, in a client machine 510, a print job 
generation unit 512 generates a print job, and stores 
it in a print job memory unit 513. Then, the print job 
25 is transmitted by a print job transmission unit 514 in 
the client machine 510 to a server machine 511 for 
managing printers . 
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In the server machine 511, a print job reception 
unit 516 receives the print job and transmits it to a 
print job interpretation unit 571. The print job 
interpretation unit 571 interprets an instruction from 
5 a user that is included in the received print job, and 
converts the instruction into information, such as a 
parameter, that can be processed by the system. When 
the parameters of the print job have not yet been set, 
the print job is transmitted to a print job automatic 

10 satup unit 517, and when the parameters have already 
been set, the print job is transmitted to a print job 
automatic change unit 518. A print job simulation unit 
601 simulates printing at the setup that is specified 
by the print job to acquire the period of time for the 

15 printing and the consumption of toner. 

The print job automatic setup unit 517 sets print 
parameters while taking into consideration the urgency, 
economy and quality values that are acquired from the 
print job. The print job automatic change unit 518 can 

20 perform the same process to change the parameters that 
have been set. The print jobs that are set or altered 
are registered in a print job schedule included in a 
print job scheduling unit 520 and are sequentially read 
to a print job transmission unit 519, or the print job 

25 is directly transmitted to the print job transmission 

unit 519 from the print job automatic setup unit 517 or 
the print job automatic change unit 518. The printing 
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is then performed. 

The status, such as printing, normal or out of 
paper, of an available printer is acquired by a printer 
state acknowledgement unit 515, and is transmitted to 
5 the print job automatic setup unit 517 and the print 

job automatic change unit 518 where it is employed for 
the setup or change of the print job. Similarly, the 
printing schedule for the print job included in the 
print job scheduling unit 520 is also transmitted to 

10 these units 517 and 518 for their employment. 

As is shown in Fig. 60, a print job may be 
transmitted along a different route. When a machine 
for generating a print job is the same as a machine for 
managing printers, the print job may not be transmitted 

15 via the print job transmission unit 514. 

Fig. 61 is a flowchart showing the processing 
performed by the print job simulation unit 601. First, 
at step S611, the print job simulation unit 601 
initializes, in preparing for the simulation, a factor 

20 for determining the printing period and the consumption 
of toner. At step S612 the period of time for the 
printing is simulated at the setup that is also 
specified by the print job, and at step S613 the 
consumption of toner is simulated at the setup 

25 designated by the print job. In this manner, the 

period of time for the printing and the consumption of 
toner are acquired. 
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A specific method is the same as that explained in 
the seventeenth and the eighteenth embodiments. 
[ Twenty- second Embodiment ] 

In this embodiment, a plurality of output trays 
5 are provided for a printer, and a tray is selected in 
consonance with a user. 

Fig. 62 is a flowchart showing the processing for 
a printer system in this embodiment. When a printer 
system receives a print job, first it performs a 
Iq 10 process for coping with a case where a plurality of 

k Q print jobs are received at the same time, and then it 

CO 

q initiates the processing shown in Fig. 62. 

Figs. 63A and 63B are diagrams showing the 
[ s external appearances of the printer systems in this 

15 embodiment. As is shown, printers 631 and 632 each 
*N have a plurality of output trays: for the printer 631, 

"J each output tray is provided with a screen for 

displaying the name of the person to whom a document on 
the tray belongs; and for the printer 631, the names of 
20 the persons whose document remain on which trays are 
displayed on a common screen. When a user is set in 
advance for each output tray, the individual trays are 
user dedicated trays* When the trays are not 
specifically assigned, the users of the output trays 
25 are changed as needed. 

The printer system in this embodiment implements 
the functions of notifying a user of the receipt of a 
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document, the termination of printing, and the failure 
to collect a printed document (uses a sensor). 

The above processing will now be described- In 
Fig. 62, at step S621 information concerning a 
5 destination is acquired from information included in a 
print job. The information concerning the destination 
is obtained, as is explained in the twenty-ninth 
embodiment, either by interpreting the setup described 
in the print job, or by extracting it from information, 

10 such as E-mail, that is set separately from the 

contents of the print job. At this time, when, as in 
conventional use, a printer is employed as an output 
device for a personal computer, usually an addressee is 
not designated. When the printer is employed as a 

15 facsimile machine or for transmission of E-mail, as in 
the system in this embodiment, an addressee is normally 
designated . 

At step S622 a check is performed to determine 
whether or not an addressee is designated. At step 

20 S623 the addressee is notified of the arrival of a 

document. At step S624 the addresser information is 
acquired from the information included in the print 
job. The addresser information is acquired, as is 
explained in the twenty-ninth embodiment, either by 

25 interpreting the designation described in the print 

job, or by extracting it from information, such as E- 
mail, that is set separately from the print job. When, 
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as in conventional use, a printer is employed as an 
output device for a common personal computer, normally 
the addresser falls within the control sphere of the 
system. However, when the printer is used as a 
5 facsimile machine, as in the system in this embodiment, 
usually the addresser is not within the control sphere. 
At step S625 a check is performed to determine whether 
the addresser is in the control sphere. At step S626 
when the addressee is designated, a corresponding 

10 output tray is selected. When the addressee is not 
specified and only the addresser is specified, an 
output tray corresponding to the addresser is selected. 
At step S627 the addresser or the addressee information 
is transmitted to the corresponding output tray, as is 

15 shown in Fig. 63A. Or, as is shown in Fig. 63B, the 
addresser or the addressee information and the output 
tray that is employed are displayed on the common 
display. When the user is set for each output tray, 
the individual trays are user dedicated trays. When 

20 the trays are not specifically assigned, the users of 
the output trays are changed as needed. 

At step S628 printing is executed. At step S629 a 
check is performed to determine whether the printing 
has been terminated and whether the addresser is within 

25 the control sphere. At step S630 the addresser is 

notified of the termination of the printing. At step 
S631 a check is performed to determine whether a 
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document: has been left; for a predetermined period of 
time and whether the addresser is in the control 
sphere. At step S632 the addresser is notified that a 
printed document has been left in the tray for an 
5 extended time period. 

[ Twenty- third Embodiment ] 

In this embodiment, information concerning the 
time is handled. Fig. 65 is a diagram showing the 
functional arrangement of a system according to the 
Ed 10 embodiment. A processor 650 includes an understanding 

pi 

unit 652, a planning unit 653, an execution unit 654, a 

iy 

□ response unit 655 and a knowledge base 656, and is 

hi connected to a database 651. 

yL Fig. 64 is a flowchart showing the processing 

Lr; 15 performed by the functional arrangement in Fig. 65 in 

~C this embodiment. 

^ At step S640 a check is performed to determine 

whether or not there is external input. If so, at step 
S641 the understanding unit 652 employs the knowledge 

20 base 656 to analyze an input document as the units of 

words and sentences, and at step S643 the object of the 
input contents is analyzed and ascertained. At step 
S644 a check is performed to determine whether 
information required for an understanding of the object 

25 is insufficient. If the information is unsatisfactory, 
at step S645 a query is issued to a user, and a 
response from the user is studied in the knowledge base 
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656. Program control thereafter returns to step S643. 
If the information is adequate, program control moves 
to step S646. When, at step S640, there is no input, 
at step S642 a job to be executed is found and program 
5 control advances to step S646. 

At step S646 the planning unit 653 prepares a plan 
to achieve the object. At step S647 a check is 
performed to determine whether information required for 
planning is insufficient. If the information is 
£0 10 unsatisfactory, at step S648 a query is issued to the 

user, and a response from the user is studied in the 

0 knowledge base 656. Program control thereafter returns 
hj to step S646. If the information is adequate, at step 
^ S649 the execution unit 654 executes the plan. At this 

1 ii 

pfi 15 time, the execution unit 654 accesses the database 651 

? or communicates with another application, as needed. 

"~ 4 At step S650 the response unit 655 determines whether 

or not a response should be made. If a response should 
be made, a decision is made concerning the selection of 
20 the contents for the response. Further, at step S651 a 
response is prepared in accordance with the selected 
contents, and is transmitted to the user. 

The understanding unit 652 acknowledges, as 
needed, externally input information, such as keyboard 
25 input, voice input, E-mail or facsimile or news 

information, analyzes the received information and also 
the time concept contained in the words in sentences, 
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and obtains an understanding of the contents of the 
sentences that are concerned with time. 

Fig. 67 is a diagram showing an example where 
schedule information is extracted from electronic mail. 
5 In the above process, specifically, as is shown in Fig. 
67, the contents of a document received by E-mail are 
analyzed, and the time concepts such as "February 
22nd", "13: 00" and "15:00" are found and analyzed. 
Then, an understanding of the sentence concerning the 

10 concept, "We have scheduled a patent system explanation 
meeting at conference room B from 13:00 to 15:00 on 
February 22" is obtained, and an action associated with 
the time can be correlated with the schedule. 

The analyzed time concept is compared with the 

15 current time. If the time concept describes the 

future, a corresponding action is registered in the 
schedule, but if the time concept describes the past, 
it is abandoned without being registered or it is 
registered as an information of a type other than that 

20 for a schedule, such as information for a personal data 
base, that is used to record past events. 

The understanding unit 652 analyses the contents 
of externally input information, such as keyboard 
input, voice input, E-mail, facsimile or news 

25 information. When the understanding unit 652 finds in 
the sentences a term (a word or a concept) that is 
unknown to the system, or a plan that the system has 
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not yet executed, and when the word or the plan that is 
detected is not urgent, the understanding unit 652 
studies the term or the plan by sequentially searching 
for information concerning it, and stores it as 
5 knowledge in the knowledge base 656. 

Specifically, when, for example, the unknown term 
XYZ appears, the presence of XYZ is memorized, and 
sentences "XYZ is very delicious" and n XYZ is hard 11 are 
analyzed to extract the concept that XYZ is hard, 

10 delicious food. 

Similarly, when a new plan is instructed and when 
from the current situation it is understood to be a 
plan for use when a user is absent, the understanding 
unit 652 studies the plan as an example of how to make 

15 a plan for when a user is absent. 

In addition, when, for example, a system has been 
acquiring information concerning a user from news and 
notifying the user each time such information is found, 
and then upon the receipt of a notification that the 

20 user responds by sending the system an instruction in 

natural language that " from now on, there is no need to 
take notice of this information" the system can 
ascertain that transmission of the information is no 
longer necessary. 

25 As is described above, in the system, the 

understanding unit 652 analyzes the contents of 
externally input information and also analyzes the 
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object. The planning unit 653 prepares a plan of an 
action for the system to take to achieve the object. 
The execution unit 654 performs the processing. And 
the response unit 655 notifies a user of the results of 
5 the processing. 

When, for example, it is ascertained, from 
information concerning an addresser, upon the receipt 
of the E-mail message shown in Fig. 67 that the mail is 
from an addresser who is not registered in the 

10 database, information concerning the addresser is 

extracted from the message and is registered in the 
personal database in the database 651. Further, when 
more information is obtained concerning the addresser 
who is not registered in the personal database, the 

15 information may be registered in the personal database. 

Since the understanding unit 652 understands the 
message is for a notice for an explanation meeting, it 
accesses the schedule of a user in the database 651, 
and compares new schedule data with the current 

20 schedule data. When the new schedule data and the 

current schedule data do not conflict, the new data are 
additionally registered in the schedule. However, as 
is shown in Fig. 69, when the schedule data conflict, a 
reply to the addresser and a notice for a user are 

25 prepared . 

As is described above, since this system 
understands the contents of received information, the 
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system can prepare a response in consonance with the 
situation and execute it. 

In the above explanation, the system prepares a 
reply to a user who sent, the E-mail without confirming 
5 it with a user, and transmits the reply. This is 

because the sentence, "Please contact me as soon as 
possible if there is any difficulty" is in the message, 
and as a result of the analysis of this sentence, it is 
determined that a response is very urgently required. 

10 If the urgency of a response is determined to be low 
because a date for the holding the meeting is fairly 
advanced and the attendance at the meeting is 
arbitrary, the system confirms it with a user before 
transmitting a response. In other words, the system 

15 determines in which action to take in consonance with 
the degree of urgency. 

In addition, the schedule or the personal database 
can be accessed to transmit a response, relative to the 
input of the natural language by a user, as well as a 

20 natural language instruction received from a user. 

For example, the personal database can be accessed 
by asking the natural language question, "What is the 
telephone number of Mr. XX?". Or the schedule can be 
accessed when the natural language question, "Where 

25 will the meeting on the 16th be held?" is received, to 
send a reply to a user. 

Fig. 66 is a diagram showing the input/output 
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types between the system that performs the overall 
processing in this embodiment and external devices. 

Input data can be data input via a keyboard, 
natural language information received by E-mail, 
5 documents or pictures input as images by a scanner, 
voice input via a microphone or images input by a 
camera. A character recognition process is preformed 
for a document that is read by the scanner, or a voice 
recognition process is performed for speech, so that 

10 input information in natural language can be obtained. 

As for when there is no input (idle state), as is 
shown at step S642 in Fig. 64, when there is no 
external input the system searches for a job that it 
should execute and regards it as input. In the idle 

15 state, when no job is given to the system, the system, 
for example, accesses news and aggressively acquires as 
input information concerning the user. 

As for output, there are the registration of a 
schedule in a database, the filing or erasure of data 

20 relative to a file memory device, or the dispatch of a 
notice to a user or a reply to an addresser. The 
importance of the output is that it can be determined 
that a process can not be performed within the control 
range of a system, and another action, such as 

25 transmitting a response to that effect, can be 
performed . 

An input/output destination is a user or an 
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external device, or a different processor in the system 
or another application. 

In addition, input documents are analyzed, and 
data concerning a person and data concerning the time 
5 are extracted and stored in the personal database and 
the schedule database. Also, data required for the 
anticipation of an action is extracted to use for 
planning . 

In the example in Fig. 67, as a result of analysis 
10 of an E-mail document that is input, the following 
description concerning a person is acquired: 
To : toshimaQabc . canon . co . j p , 

rohraQabc , canon .co.jp, 

kazuyoQabc . canon .co.jp 
15 From ichiro@abc.canon.co.jp 

I am Suzuki of system promotion section. 
Suzuki ( ichiroQabc . canon . co . j p ) 
044-123-4569 (ext. 654-3210) 

Canon Inc. Intellectual Property Head Office System 

20 Promotion Section 

Assuming that a person whose mail address is 
" kazuyo@abc . canon .co.jp" is already registered in the 
personal database and the other persons are not 
registered, as is shown in Fig. 67, data for PERSON 1, 

25 2 and 3 are extracted and registered in the database 
651. 

Since there is a description concerning the time, 
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"We have scheduled a patent system explanation meeting 
at conference room B from 13:00 to 15:00 on February 
22" data for EVENT1 is extracted. 

Further, since there is a description used for 
5 estimating an expected action, "Please contact me as 

soon as possible if there is a difficulty" this is used 
for the processing to be explained while referring to 
Fig. 70. 

Fig. 68 is a specific flowchart for step S649 

10 prepared by extracting the portions from the flowchart 
in Fig. 64 that are required for performing the 
processes in Figs. 67 and 69. 

The processing will now be explained while 
referring to Figs. 67 and 69. 

15 At step S680 E-mail is input and at step S681, of 

each word and sentence, the input E-mail document is 
analyzed . At step S683 it is ascertained from a 
signature and a header that "there is information 
concerning persons" and "an event, an explanation 

20 meeting, will be held" . Further, it is ascertained 

that "a reply is requested if there is a difficulty". 

At step S684 the data concerning a person is 
extracted and a plan for registering the data in a 
database and a plan for registering the explanation 

25 meeting event in the schedule are made. At step S685, 
it is ascertained that there is a plan to be executed. 
At step S686 it is found that there is no problem with 
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the plan, and program control thereafter moves to step 
S688. At step S688 the data concerning the person is 
extracted and registered in the database, and program 
control returns to step S685. 
5 In Fig. 67, at step S685 there is a plan to be 

executed: registration of the explanation meeting event 
in the schedule- At step S686 it is assumed that the 
event can be registered in the schedule with no 
problem, and program control goes to step S688. At 

10 step S688 the event is registered in the schedule, and 
program control returns to step S685. At step S685 
there is no other plan to be executed, and at step S689 
no acknowledgement receipt is required- The processing 
is thereafter terminated. 

15 In Fig. 69, at step S685 there is a plan to be 

executed: registration of the explanation meeting event 
in the schedule, and at step S686 a conflict is found 
in the schedule. Since there is a problem with the 
execution of a plan, at step S687 a reply to that 

20 effect is transmitted and a plan is made to notify a 

user that there is a problem. At step S685 there is a 
plan for writing a reply and transmitting it, and at 
step S686 no problem concerning the execution of the 
plan is found. At step S688, therefore, a reply 

25 indicating that the timing is inconvenient is prepared 
and transmitted to the addresser. At step S685 there 
is a plan for preparing a document for notification of 
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a user. At step S686, there is no problem in 
execution, and at step S688 a response to a user is 
prepared describing that a reply was sent for re- 
consideration because of the contents of the E-mail and 
5 the conflict of the schedules. At step S685 there is 
no plan to be executed and at step S689 since there is 
an notice to a user, it is transmitted to the user. 
The processing is thereafter terminated. 

In the example in Fig. 69, when the schedule 

10 information EVENT1 that is extracted is compared with 
the schedule EVENT2 that is planned previously, it is 
found that the two events conflict. Further, according 
to the information extracted in the example in Fig. 69, 
it is apparent that a reply is necessary if there is a 

15 difficulty. In the system in this embodiment, the 

knowledge for preparing letter is employed to write a 
letter describing that a user can not attend the 
meeting because the schedules are conflicting, and the 
reply is automatically transmitted. 

20 In addition, the user of the system in this 

embodiment is notified that the system automatically 
has transmitted a reply for the user. 
[Twenty- fourth Embodiment] 

Fig. 70 is a diagram showing an example where a 

25 user sends a question by voice to a system according to 
this embodiment. 

When a user asks by voice "where will tomorrow's 
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conference be held?", the system can examine the 
schedule information of the user and tell the location 
of the conference. 

When the user does not know how to get the 
5 location for the conference and asks "In which area in 
Shimomaruko? " , the system examines the database, 
performs planning for an explanation to make a user 
understand, prepares a briefing story, and uses 
graphical images and sounds to explain how to get the 
10 location. 

Fig. 71 is a flowchart showing the processing 
performed in Fig. 70 by referring to the basic 
flowchart in Fig. 64. An external input is regarded as 
a query . 

15 The processing relative to a first inquiry will be 

explained. At step S710 voice (where will tomorrow's 
conference be held?) is externally input, and at step 
S711 the input sentence is analyzed. At step S712 it 
is understood that the input sentence is an inquiry for 

20 the conference, and that the object of the user is to 
know the location of the conference. At step S713 the 
following planning is performed to answer the inquiry. 
( 1 ) Tomorrow 1 s schedule is extracted . ( 2 ) The 
location registered in the schedule is acquired. (3) 

25 A reply is prepared to tell the acquired location. 

At step S714 information is sufficient and program 
control moves to step S716. If the schedule show a 
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plurality of conferences, the system does not identify 
which conference is. Therefore, at step S715 a query 
is sent to a user to remake a plan. At step S716 the 
plan made at step S713 is executed, and a reply 
5 document to the user is prepared. At step S717 it is 
ascertained that there is a response to the user, the 
system determines the transmission of the response. At 
this time, since it is understood that the input was 
done by voice, transmission of the reply by voice is 
10 also determined. At step S718 the response, which is 
voice data obtained by conversion, is transmitted to 
the user. 

The processing is temporarily terminated, and 
following this, an inquiry from the user is input. The 

15 process relative to the second inquiry will now be 
described. 

At step S710, voice (In which area in 
Shimomaruko? ) is input externally, and at step S711 the 
input sentence is analyzed. At step S712 it is 

20 understood that this inquiry is a continuous question 
of the first one, and also that a specific location of 
the place name that was answered is being asked. 

At step S713 the following plans are made to 
respond the question: (1) a plan for explaining that 

25 it is difficult to explain the location by writing; (2) 
a plan for explaining the location by sequentially 
sending images; (3) a plan for acquiring necessary 
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images from an image database and a location database; 
and ( 4 ) a plan for preparing sentences corresponding to 
images . 

At step S714 the information is adequate, and at 
5 step S716 the plans made at step S713 are sequentially 
executed and a reply to a user consonant with the 
images is prepared. At step S717, since there is a 
reply document to be transmitted to the user, the 
transmission of the response is determined. At this 
10 time, it is understood that the input was done by 

voice, so that it is also determined that explanation 
will be given by voice while displaying images on a 
monitor. At step S718 the response is transmitted to 
the user. The processing sequence in Fig. 70 is 
15 thereafter terminated. 

[ Twenty- f i f th Embodiment ] 

In an example shown in Fig. 72, when two scheduled 
events conflict, the order of priorities of the two 
events is obtained from a comparison, and a plan to 
20 handle this problem is prepared and proposed to a user. 

That is, in this example, when one schedule is 
found to be more important than the other, it is 
proposed that the less important schedule be canceled. 
If the user knows somebody to send as his or her 
25 proxy to the scheduled event for which cancellation is 
proposed, the user can send the system a response to 
that effect. The system can then perform an action in 
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consonance with the response. Also, the system can 
study a plan for despatching another person as a proxy. 

In Fig. 72 is shown a combination of other methods 
performed at step S646 in Fig. 64 to make plans when 
5 the schedule conflict is found at step S686 in the 
flowchart in Fig. 68. 

The processing in Fig. 68 has been explained for 
unconditionally transmitting to a user a response for 
an inconvenience. In this embodiment, the processing 
10 for a plurality of plans is employed as the re-planning 
method at step S646 . 

Fig. 73 is a flowchart for evaluating a priority 
extracted from the contents of a document and for 
proposing it to a user. The processing will now be 
15 explained . 

The priorities of the scheduled events is 
determined by using information that the user has set 
in advance for each schedule, attendance at an event 
included in the schedule, the object of a scheduled 
20 event and the result obtained as a result of the 

analysis of the received document. A user may set the 
priorities of persons that are expected to attend the 
pertinent event. 

The priority determined according to the object of 
25 the schedule may be set in advance by a user, or may be 
determined while taking into account the object or the 
field that the system understands, from the study of 
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the past, that the user is interested in. That is, 
when the user engages in computer associated work, the 
schedule for a computer associated event priority over 
an event associated with real estate. Of course, if a 
5 user plans to purchase a house and that data is stored 
as information concerning the user, the schedule of an 
event associated with real estate may take priority. 

For the priority determined from the result of the 
analysis of the document, when a document including the 

10 sentence " Please be sure to attend the meeting" and a 
document including the sentence " Please attend the 
meeting if possible" setup a conflict, the first 
document received is determined to take priority. 

In Fig. 73, at step S730 the current schedule 

15 information is acquired. At step S731 the priority for 
new schedule information is evaluated. At step S732 
the priorities of both schedules are compared. At step 
S733 from the result of the comparison, a plan is made 
to prepare a notice for querying a user (e.g., display 

20 two conflicting scheduled events and ask whether the 
event having the lower priority can be canceled ) . At 
step S734 the prepared notice is transmitted to the 
user. At step S735 a response from the user relative 
to the inquiry concerning the plan is acquired. The 

25 same re-planning process as in Fig. 68 is performed to 
again prepare a plan by using the response obtained 
from the user so that an action consonant with the 
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response of the user can be taken. 

Fig, 74 is a flowchart showing the re-planning 
process . 

At step S740 the response finally obtained in Fig. 
5 73 is regarded as input. At step S741 the input 

sentence is analyzed. At step S742 it is ascertained 
that the schedule is to be changed and that this is the 
first instructed plan. 

At step S743 the following plans are made: (a) 
10 the changing of the schedule; (b) the preparation of a 
document to transmit the schedule to a proxy; (c) the 
study of a new plan; (d) the preparation of a document 
to transmit the new plan to the user; and (e) the 
notification sent to a user concerning another schedule 
15 that should be recalled. 

At step S744 the plans made at step S743 are 
executed: (a) the schedule is changed; (b) a document 
is prepared to transmit the schedule to a proxy; (c) a 
new plan is studied; (d) a document is prepared to 
20 transmit the new plan to the user; and (e) a user is 
notified of another schedule that should be recalled. 

At step S745 the notification document is 
transmitted to the user. The processing for performing 
the process in Fig 72 is thereafter terminated. 
25 [Twenty-sixth Embodiment] 

Fig. 75 is a diagram illustrating the overall 
image of a system according to this embodiment, 
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including input/output: devices. Fig. 76 is a flowchart 
showing the processing performed by the system in Fig. 
75. 

Information input by E-mail, by voice, via a 
5 keyboard, across the WWW, and by input devices, such as 
a telephone, a facsimile machine, a scanner and a 
camera, is analyzed by an input management unit 751 
(step S760). A core unit 752 ascertains the contents 
of the information and plans an appropriate process 

10 (step S761). An output management unit 753 determines 
an output medium and prepares the contents to be output 
and an output route (step S763). Then, the information 
is output by E-mail, by voice, across the WWW, or by 
output devices, such as a telephone, a facsimile 

15 machine, a printer and a copier. 

Figs. 77 to 79 are specific flowcharts showing the 
process at step S760 (the process performed by the 
input management unit 751), at step S761 (the process 
performed by the core unit 752) and at step S763 (the 

20 process performed by the output management unit 753). 

In Fig. 77, when, at step S770, it is determined 
that there is new input, at step S771 the input 
information is obtained. At step S772 the input 
information is analyzed, as needed, after the 

25 information has been identified. At this time, the 
input management unit 751 accepts information from 
various media and identifies or analyzes the data using 
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methods corresponding to the individual media. 

In Fig, 78, the core unit 752 receives the 
information that is analyzed in Fig. 77. 

At step S780 the object of the input information 
5 is analyzed and ascertained from the result of the 
analysis. At step S781 how the object of the input 
information is related to the main apparatus (the 
system or the user) is examined. At step S782, from 
the relationship with the main apparatus, planning for 
10 what to execute is performed in consonance with the 
object. At step S783 the plan made at step S782 is 
executed . 

In Fig. 79 the output management unit 753 employs 
the results obtained in Fig. 78 to determine whether a 

15 response is needed. If a response is required, it is 
prepared and output . 

At step S790 the result obtained in Fig. 78 is 
analyzed. At step S791 a check is performed to 
determine whether or not there is a response relative 

20 to the results. When there is no response, the 

processing is thereafter terminated- If there is a 
response, at step S792 a response relative to the 
external response is determined. At step S793 a medium 
for the response is determined. At step S794 a 

25 response is prepared in consonance with the designated 
medium. At step S795 the prepared response is issued 
by the designated medium. At step S896 the response is 
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actually output:. The processing performed by the 
arrangement in Fig. 75 has been explained. 

When, for example, E-mail is received that 
notifies the user of the holding of a conference, the 
5 input management unit 751 analyses the input 

information according to the procedures in Fig. 77. 
From the obtained result, the core unit 752 prepares a 
plan, according to the procedures in Fig. 78, as to how 
the user and the system should handle the information. 

10 The plan is transmitted to the output management unit 
753, which in turn performs the process according to 
the procedures in Fig. 79. When a response should be 
transmitted with the plan, a response is actually 
prepared and transmitted. 

15 Depending on the contents of the conference 

described in E-mail message and the status of the 
schedule of the user, at step S782 a plan is prepared 
for the aggressive transmission of the response to the 
user, so that preparation of the response is required. 

20 At step S791, therefore, it is determined that a 
response is to be transmitted. At step S792 the 
contents of the response are determined. When the 
schedules conflict and when it is uncertain which 
schedule should be selected or whether a user intends 

25 to attend a specific conference, the contents of a 

response asking for a decision are determined. At step 
S793 a medium is selected by which the contents of the 
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response can be transmitted most effectively. When, 
for example, the user is outside the office, a medium, 
such as a telephone or a facsimile machine, is selected 
that can issue a notice to the user. When the user is 
5 being operating a personal computer, a medium, such as 
a personal computer, is selected that can effectively 
display various reference materials. At steps S794 and 
S795 a document and an image are prepared in consonance 
with the contents and the medium that are decided 

10 above, and a response is prepared using a method for 
converting the data into voiced natural language in 
consonance with a specific medium, and is issued. 

Fig. 80 is a detailed flowchart for Fig. 75 to 
explain the processing for extracting data from input 

15 information. In this system according to the 

embodiment, when a document is input, at step S801 type 
identification symbols, such as the layout and a bar 
code of a document, are employed to infer the document 
type. If the document is inferred to be a letter, a 

20 report or a patent publication, program control moves 
to step S803 . When no identification of the document 
is possible, program control goes to step S810 whereat 
OCR is thoroughly performed to ascertain the document 
type . 

25 At step S803, in order to confirm the inferred 

document type is correct, a characteristic block is 
interpreted by the OCR to identify the document type 
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(see Figs* 82A and 82B). At step S804 the addressee of 
a letter and the patent serial number of the patent 
publication are found and the document type is 
confirmed. When the confirmed document type matches 
5 the inferred document type, program control advances to 
step S806. If the document types do not match, program 
control moves to step S810. For slips, at the top of 
which character strings describing the slip type are 
printed, the document type can be easily determined by 

10 the OCR on the employing upper portion of the sheet. 

At step S806 the knowledge base of the document 
type that has been confirmed is employed to read and 
interpret a specific block using OCR. At step S807 a 
check is performed to determine whether there was 

15 information in the past that concerns the resultant 

information. If such information was present, program 
control moves to step S808. If such information was 
not present, program control goes to step S810. As a 
result, the addressee or the patent number is found and 

20 whether the information is a reply to a letter that was 
sent in the past can be ascertained. 

At step S808, based on the identified status, the 
object is decided from the contents of the important 
text, and at step S809 a process to be executed is 

2 5 determined . 

At step S811, because of the above object, the 
information is analyzed in another range where OCR has 




not yet; been performed. At step S812 a required 
process is actually performed. As a result, a filing 
process, etc., is performed as needed. 

In the above process, the OCR can be performed 
5 while inferring the contents of the information, more 
efficient and precise OCR results can be obtained than 
when, as in the conventional case, the OCR use is 
effected from the beginning. Accordingly, the process 

_ based on the OCR results can be performed precisely. 

4f 10 Fig. 81 is a diagram showing an example 

letter/facsimile that is a target for data extraction. 

*=J In this example, at step S801 the document type is 

UJ inferred from the layout of a document. As a result, 

M DocTypel = letter/fax. 

as ; 

I Ls 

ffj 15 At step SS03 a specific block is read by the OCR 

i.g and is interpreted (see Figs. 82A and 82B). Then, 

"From", "To" and "Dear Sir" are acquired, and at step 
S804 DocType2 = letter/ fax. 

The process for reading a specific block by using 
20 OCR, which was explained at step S803 in Fig. 80, will 
be specifically described. Figs. 82A and 82B are 
diagrams for explaining this process. 

The system in this embodiment scans a specific 
block as follows: 
25 (1) A specific block is read fast by pre-scanning with 
a low resolution. 

( 2 ) A form of the information that is read is compared 
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with an information form stored in a DB 823. When the 
forms match, input information other than the form, or 
information in an area (e.g., inside the frame of a 
card in Figs. 82A and 82B) specified for each form, is 
5 read at a high resolution. 

(3) Only the area that is read is regarded as an OCR 
target and analyzed, and the processing is continued. 

In addition, since in this example it is assumed 
that there are a plurality of cards having the same 

10 form, documents that are to be read at a predetermined 
interval probably have the same form. Therefore, 
first, the process is performed while it is assumed 
that the documents have the same form, and when a 
contradiction occurs, it is assumed that the documents 

15 have different forms and a re-analysis of them is 
performed. 

As a result, the processing speed is considerably 
increased, the analysis range is limited, and an 
analysis domain is determined by specifying a form, and 
20 the analysis precision is drastically improved. 

The determination of the object from the contents 
of the information, which was explained at step S808 in 
Fig. 80, will be specifically described. 

The system in this embodiment employs date data to 
25 perform the determination process according to the 

procedures in Figs. 83A and 83B. When a sender is a 
user, the following process is performed. 
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(1) When the current date is today, transmission by 
facsimile is decided. 

( 2 ) When the date is yesterday or some time in the 
past, the user is queried as to whether the same 

5 document was received before or whether he or she has 
seen the same contents, in order to determine whether 
to perform filing, re-transmission, or whether the user 
made a mistake. 

(3) When the date is tomorrow or some day in the 
10 future, the user is queried in order to determine 

whether the information should be held until the 
designated date, or whether the user made a mistake. 

More specifically, assuming that in a document in 
Fig. 81, Datel is the date of a document, Date2 is the 
1 5 today 1 s date , 

(1) Datel = November 25, 1996 & Date2 = November 25, 
1996 send the document 

(2) Datel = November 25, 1996 & Date2 = July 2, 1996 - 
- filed 

20 (3) Datel = November 25, 1996 & Date2 = November 26, 
1996 

Date of contents = December 18 and 22, 1996 
mistake 

Figs. 83A and 83B are flowcharts showing the 
25 processing for analyzing the object extracted from an 

input document. When a document is input, at step S830 
a check is performed to determine whether the sender is 
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a user. If the sender is a person other than a user, 
the input document is determined to be a received 
document. At step S831 a corresponding object, such as 
filing or data extraction, is examined, and the 
5 processing is thereafter terminated. 

When the sender is a user, program control 
advances to step S832 to analyze the type of the input 
document. At step S833 the date of the document is 
compared with the current date. When the two dates are 

10 close to each other, at step S835 a check is performed 
to determine whether the document was transmitted 
before. When the document was not transmitted before, 
its object is determined to be 11 transmission of a 
document". If the same document was transmitted 

15 before, at step S843 the object for filing or re- 
transmission is extracted. 

When the date of the document is a fairly old 
date, program control moves to step S836, whereat a 
check is performed to determine whether or not the same 

20 document was transmitted before. When the document was 
transmitted before, program control moves to step S843. 
When the document was not transmitted, program control 
goes to step S837 whereat a check is performed to 
determine whether any other date is described in the 

25 information for the document. When there is another 

date, program control goes to step S843. When no other 
date is found, it is ascertained that the object can be 
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transmission of the document, and also that the date 

may be wrong. 

When the date of the document is a date fairly far 

in advance, at step S840 a check is performed to 
5 determine whether any other date is described in the 

information for the document. When, at step S841, 

there is another date, the object is determined to be 

"transmission of the document". When no other date is 

found, program control moves to step S839. 
10 [Twenty- seventh Embodiment] 

Fig. 84 is a diagram showing an example 

arrangement of a system according to this embodiment. 

In this embodiment, the system serves as a current 

facsimile machine . 
15 According to the system in this embodiment, 

without a user specifying an addressee, the addressee 

for a document is determined from bar code on a 

document that is read or information on a cover sheet, 

so that the document can be transmitted to a correct 
20 addressee. 

As a result, appropriate information can be 

transmitted to a printer or a facsimile machine, by E- 

mail or via the WWW. 

[ Twenty-eighth Embodiment ] 
25 Fig. 85 is a diagram illustrating an example 

arrangement of a system according to this embodiment. 

In this example, the system implements a desk sorter 
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function for processing all the kinds of documents on 
the desk. 

That is, in consonance with the contents of the 
document that is read, sorting, filing, scheduling, 
5 data extraction and automatic processing are performed* 
[Twenty-ninth Embodiment] 

Fig. 86 is a detailed diagram for explaining the 
system in Fig. 85. The processing performed by the 
system shown in Fig. 86 will be specifically described 

10 while referring to the flowchart in Fig. 87. 

Fig. 87 is a flowchart showing the processing for 
this embodiment. Figs. 88 and 89 are tables for 
knowledge used during the processing; the table in Fig. 
88 shows the knowledge in general knowledge base, and 

15 the table in Fig. 89 shows the knowledge in a knowledge 
base of a field specified on a cover sheet. In this 
embodiment, information defined in the tables in Figs. 
88 and 89 is employed to compare a character string 
included in an input document with a character string 

20 defined in a character string column in the table, and 
the conception in a corresponding column, a Role for 
further specifying the meaning, and a Condition for 
specifying succeeding information, or an instructed 
process Action are acquired, and analysis for them is 

25 performed. 

In Fig. 87, at step S870 a cover page is scanned 
and an OCR process is performed. At step S871 the 
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knowledge base in Fig. 88 is examined to extract from 
it data, such as the names of a sender and a receiver 
and their telephone numbers and facsimile numbers. At 
step S872 the obtained data are registered in a 
5 database. In the example in Fig. 86, character string 
"To: Macrohard Corp." is included in the cover page of 
a received document. When this character string is 
compared with an item defined in the character string 
column in Fig. 88. Then, matching item "To 11 is found 

10 and its Role is a reception company name or an 

individual name, so that "Macrohard Corp." is extracted 
as information concerning a receiver. 

At step S873 a notification method and means, a 
filing job, an action for a place, and a job are 

15 extracted. In the example in Fig. 86, character string 
"File: MH/Contract" is included in the cover page of 
the received document. When this character string is 
compared with the items defined in the character string 
column in Fig. 89, matching item "File" is found, 

20 filing is extracted from the column as an instructed 
action, and "MH/Contract" is extracted from the 
document as information for a filing place. 

Assuming that Datel = November 25, 199 6 and Date2 
= November 25, 1996, as is explained in Figs. 83A and 

25 83B, it is determined that the object is transmission 
of a document. 

At step S874 a check is performed to determine 
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whether an action for the object of the processing is 
present. Since the transmission of the document to a . 
receiver is the object, at step S875 a transmission 
action is performed. At step S876 the information on 
the cover page is employed to determine whether the 
filing is necessary. Since the filing is required in 
the example in Fig. 86, at step S877 the document is 
filed at the instructed place, "MH/Contract" . 

At step S878 as well as at step S876 the 
information on the cover page is employed to determine 
whether or not a notification for the transmission of 
the document should be issued to the sender. Since it 
is instructed to notify the sender, by voice, of the 
transmission of the document, at step S879 the sender 
is notified by voice of that the document has been 
transmitted to the receiver. At step S880 whether or 
not another action is determined. At step S881 a 
keyword is designated, and an index is prepared by 
using keyword ABC. 
[Thirtieth Embodiment] 

Fig. 90 is a diagram showing an example processing 
where the history in the past is referred to based on 
an instruction by voice in natural language. The 
processing in Fig. 90 will be explained while referring 
to the flowchart in Fig. 91. 

Fig. 91 is a flowchart showing the processing for 
this embodiment. When an aural instruction "Fax 
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Contract again to John, Notify him by Phone" is 
received from a user, the system analyzes the object of 
the instruction in the natural language, and 
understands "Re-transmit document "Contract" to John, 
5 and Notify him of the transmission". Thus, the 

processing in Fig. 91 is initiated to specify the 
document Contract and the addressee John. 

Since re-transmission means that the document was 
previously transmitted, at step S910 history 

10 information is acquired from the database. Then, the 
document Contract and the addressee John are specified 
from the history information. At step S911 the address 
of the John is obtained from the personal database. 
Since, in the database, there are two Johns, John Smith 

15 and John Bush, ordinarily the addressee can not be 

specified. Since at step S910 the history information 
is referred to, however, it is judged that John to whom 
the document Contract was transmitted is John Smith, 
and the addressee is thus specified. At step S912 the 

20 document "Contract" is acquired from the filing 

database. At step S913 the document is transmitted to 
addressee John. Since there is notice instruction 
"Notify him by Phone" at step S914 it is determined 
that transmission of the notice is necessary. At step 

25 S915, as well as in Fig. 89, according to the rule of 
the knowledge base, an instruction is issued to a 
facsimile machine 901 as a reception apparatus to send 
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a notice "to a receiver by phone. At: step S916 the 
other actions are not instructed, and program control 
thereafter determined . 

Upon the receipt of the document, according to the 
5 notification instruction, the facsimile machine 901 

notifies John by phone of the receipt of the document 
from Mr. Doors. When the facsimile machine 901 can not 
notify the addressee by phone of the receipt of the 
document, the addressee may take its place. 

10 [Thirty-first Embodiment] 

Fig. 95 is a flowchart showing example processing 
where, from the contents of a document that is input 
and analyzed, it is determined that a user should do 
something, a query is actually transmitted to the user, 

15 and a required process is preformed automatically. 

Fig. 92 is a diagram showing an example input document. 
Fig. 93 is a diagram showing the contents of a 
conversation between a system and a user. Fig. 94 is a 
diagram showing a document that is prepared by the 

20 system as the result of judgement of the conversation 
in Fig. 93. 

The processing for preparing an output document in 
Fig. 94 from an input document in Fig. 92 will be 
described while referring to the flowchart in Fig. 95. 
25 At step S950 the input document in Fig. 92 is 

analyzed and the following objects are understood: (a) 
Mr. John Smith arrives at Yokohama at three o'clock on 
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November 28; (b) Mr. John Smith little understands 
Japanese; and (c) some one should meet him at the 
station. 

At step S951 what the user should do is determined 
5 from the contents that are obtained. Since it is 

determined that some one must meet him at the station, 
program control advances to step S952. At step S952 a 
check is performed to determine whether a proxy 
(system) may execute the action. Since the system can 

10 not meet him at the station, program control goes to 
step S953. At step S953 a check is performed to 
determine whether an inquiry should be issued to the 
user, and if so, program control moves to step S954. 
In this example, since it is determined from the 

15 addressee of the input document that this is a request 
for Dr. Aruna Rohra, program control moves to step 
S954. 

At step S954 a process for inquiring the user (Dr. 
Aruna Rohra) is performed. In this case, as the result 

20 obtained at step S951 the user must meet John at the 
station, and an inquiry what to do is issued to the 
user. At step S955 what to do next is determined from 
the result of the inquiry, and whether or not an action 
is to be initiated is determined. In this example, the 

25 user instructs to ask Mr. Tanaka to meet him, the 
action is initiated according to the instruction. 
Since there is a request for Mr. Tanaka to act as a 
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proxy, it: is assumed -that the action must be initiated. 
At step S956 a document for requesting Mr. Tanaka as a 
proxy is prepared and a contact is made with Mr. 
Tanaka. In this example, an electronic mail document 
5 is prepared and transmitted. In this example, 

information that John arrives on 28th, an instruction 
for meeting him for the user and the original document 
that caused the request are attached to automatically 
form the document. 

10 As is described above, processing has been 

explained for performing the matter (meeting John at 
the station) where from the input document the user 
must actually perform an action. 
[ Thi r ty- second Embodiment ] 

15 Fig. 96 is a diagram illustrating a thirty-second 

embodiment where a system is operated by a controller. 

According to the system in this embodiment, 
devices and a controller directly or indirectly 
communicate with each other to specify the other. 

20 Therefore, the following functions are provided: 

(1) automatic specifying of a device; (2) acquisition 
of a device capability from the device; (3) a UI (User 
Interface) inherent to a device; (4) operation by 
voice; (5) common basic operation for all the devices; 

25 (6) a UI inherent to a user; and (7) a UI having a high 
degree of freedom in consonance with the status. 

That is, (1) only when a controller faces an 




- 125 - 



operating target device, it automatically recognizes 
the target device; (2) information for each device is 
acquired from the pertinent device via IrDA or via a 
wireless LAN; (3) a UI inherent to each device is 
5 provided from the information for the device; (4) an 
operation by voice can be performed; (5) common basic 
operation can be provided by using the same controller; 
(6) user identification information is employed to 
acquire necessary information for each user from the 

10 database or the address book and to implement the UI 
inherent to the user; and (7) the optimal UI is 
provided in consonance with the status. 

In Fig. 96 since a controller 960 faces a 
facsimile machine 963, it acquires model identification 

15 information from the facsimile machine 963, and renders 
a UI, which corresponds to the model, active. Then, 
following the displayed "Fax to" character string "Fax 
to John" is input with a touch pen, and transmitted to 
the facsimile machine 963. John's facsimile number is 

20 read from an address book 965 in a personal computer 

964, and a document set in the facsimile machine 963 or 
a document designated on a file in the personal 
computer 964 is transmitted to the fax number. The 
detailed processing will be described later. 

25 [Thirty-third Embodiment] 

Fig. 97 is a diagram showing a status monitor. 
In a system according to this embodiment, 
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information of an apparatus that is other than the 
apparatus that a user directly operates can be referred 
to. 

Therefore, the following functions can be 
5 provided : ( 1 ) remote and handy management ; ( 2 ) 

specifying of automatic device; (3) common input/output 
for examining a status; (4) the same function as a 
portable telephone; (5) operation by voice; and (6) 
handling of all the devices that can input/output 

10 infrared rays. 

That is, (1) required information can be acquired 
even from a remote area from a target device; (2) only 
when a controller faces the target device, it can 
automatically recognize the target device; (3) common 

15 basic operation can be provided by using the same 

controller; (4) the usability is enhanced by providing 
the portable telephone function for the controller; (5) 
operation by voice can be performed; and (6) the common 
protocol is employed to cope with all the devices that 

20 can input and output infrared rays. 

In Fig. 97, a status monitor 970 selects a printer 
961, a copier 962 or a facsimile machine 963, and 
character string "Status" is input with a touch pen and 
instructed. The status monitor 970 retrieves status 

25 information from the device that the status monitor 970 
faces, so that the user of the status monitor 970 can 
confirm the statuses of the devices. Even when a 
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personal computer 964 is located at an area where a 
command from the status monitor 970 can not reach, the 
status of the personal computer 964 is requested to a 
printer 961, which can communicate with the personal 
5 computer 964 via the network. Then, the status 

information can be retrieved to the status monitor 970 
from the personal computer 964 via the network and the 
printer 961. 

[ Thirty- fourth Embodiment ] 

10 Fig. 98 is a diagram showing an example where a 

controller 960 recognizes a target model, and a 
corresponding UI is read from the controller 960 and 
displayed. The controller 9 60, which does not face a 
target device, has a default screen on which message 

15 "Select a device which you would like to use" is 
displayed, as is shown in the center in Fig. 98. 

The communication between the controller and the 
target model will now be described. It should be noted 
that (2) and (3) are not necessarily required. When, 

20 for example, in (4) a target model generates a signal 
to the controller at predetermined intervals, the same 
effect can be obtained. The positional relationship 
between the controller 960 and a target model may be 
employed to determine that the controller 960 selects 

25 the target model. Further, in (4) the controller 960 
may identify a target model by examining the shape of 
the model, or by reading the bar code attached to the 
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target model. In other words, only the essential 
portion is shown in Fig. 98. 

(1) A user directs the controller 960 to a target 
model . 

5 ( 2 ) The user touches a touch panel on the controller 
960 in order to determine the timing at which the 
controller 960 transmits to the target model a target 
model identification signal request signal. Or, the 
controller 960 is set in advance to transmit a target 
10 model identification signal request signal at a 
predetermined interval . 

(3) The controller 960 transmits a target model 
identification signal request signal to the target 
model . 

15 (4) A target model identification signal is 

transmitted upon the receipt of the target model 
identification signal request signal from the 
controller 960, or according to the setup that a target 
model identification signal request signal at a 

20 specified interval. 

(5) The controller 960 that receives the target model 
identification signal determines a UI in consonance 
with a target model indicated by the signal, and 
displays a corresponding UI internally stored. In this 

25 case, the identification signal is provided for each 
model; however, when an identification signal is 
provided for each device, a different UI can be 
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employed for each of devices though they are the same 
model • 

Fig. 99 is a flowchart showing the processing 
performed by the controller 960 when the controller 
recognizes a target model, and reads and displays a 
corresponding UI. The processing performed by the 
controller 960 will now be described. 

At step S991 the controller 960 waits until an 
instruction from a user is input. At step S992 a 
target model identification signal request signal is 
transmitted by an infrared ray communication method in 
order to obtain a target model identification signal 
from the target model. At step S993 the controller 960 
waits until the target model outputs a target model 
identification signal. At step S994, the UI 
corresponding to the received target model 
identification signal is acquired from internal memory 
and is displayed. 

Fig. 100 is a flowchart showing the processing 
performed by a target model when the controller 960 
recognizes a target model, and reads and displays a 
corresponding UI . The processing performed by the 
target model will now be described. 

At step S1001, the target model waits for a 
request signal from the controller 960. At step S1002 
the target model transmits a target model 
identification signal by an infrared communication 
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method , etc . 

[ Thirty- f i f th Embodiment ] 

Fig. 101 is a diagram showing an example where a 
controller 960 receives a UI from a target model and 
5 displays it. The controller 960, which does not face a 
target device, has a default screen, as is shown in the 
center in Fig. 101. 

The communication between the controller and the 
target model will now be described. It should be noted 

10 that (2) and (3) are not necessarily required. When, 
for example, in (4) a target model generates a signal 
to the controller at predetermined intervals, the same 
effect can be obtained. The positional relationship 
between the controller 960 and a target model may be 

15 employed to determine that the controller 9 60 selects 
the target model. Further, in (4) the controller 960 
may identify a target model by examining the shape of 
the model, or by reading the bar code attached to the 
target model. In other words, only the essential 

20 portion is shown in Fig. 101. 

(1) A user directs the controller 960 to a target 
model . 

( 2 ) The user touches a touch panel on the controller 
960 in order to determine the timing at which the 

25 controller 960 transmits a UI request signal to the 

target model. Or, the controller 960 is set in advance 
to transmit a UI request signal at a predetermined 
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interval . 

(3) The controller 960 transmits a UI request signal 
to the target model . 

(4) A UI signal is transmitted upon the receipt of the 
UI request signal from the controller 960, or according 
to the setup that a UI request signal at a specified 
interval . 

(5) The controller 960 that receives the UI signal 
displays the UI . 

Fig. 102 is a flowchart showing the processing 
performed by the controller 960 when the controller 
receives a UI from a target model and displays it. The 
processing performed by the controller 960 will now be 
described. 

At step S1021 the controller 9 60 waits until an 
instruction from a user is input. At step S1022 a UI 
request signal is transmitted by an infrared ray 
communication method in order to obtain a UI signal 
from the target model. At step S1023 the controllier 
960 waits until the target model outputs a UI signal, 
at step S1024, the received UI is displayed. 

Fig. 103 is a flowchart showing the processing 
performed by a target model when the controller 960 
receives a UI from a target model and displays it. The 
processing performed by the target model will now be 
described. 

At step S1031, the target model waits for a 
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request signal from the controller 960. At step S1032 
the target model transmits a UI signal by an infrared 
communication method, etc. 
[Thirty- sixth Embodiment] 
5 Fig. 104 is a diagram showing an example where UIs 

of devices are stored in a wireless LAN server 1041 and 
when a controller 960 faces a copier 962 as a target 
model and requests a UI, a UI is transmitted, upon a 
request from the copier 9 62, from the wireless LAN 
10 server 1041 to the controller 960, and is displayed. 

The controller 960, which does not face a target 
device, has a default screen, as is shown in the center 
in Fig. 104. 

The communication between the controller and the 
15 target model will now be described. 

(1) A user directs the controller 960 to a target 
model (a copier 9 62). 

(2) The user touches a touch panel on the controller 
960 in order to determine the timing at which the 

20 controller 960 transmits a UI request signal to the 

target model. Or, the controller 960 is set in advance 
to transmit a UI request signal at a predetermined 
interval . 

(3) The controller 960 transmits controller 

25 identification data (controller A) to identify itself 
and a UI request signal to the target model . 

(4) When the target model receives the controller 




- 133 - 



identification data and the UI request signal from the 
controller, the target model transmits, to the server 
1041 that manages the target model, the controller 
identification data (controller A) and target model 
5 identification data (copier) to identify itself, and 
requests the transmission of the UI . 

(5) The server 1041 transmits the UI of the target 
model, which is indicated by the target model data, via 
a wireless LAN to the controller 960, which is 

10 indicated by the controller identification data. 

(6) The controller 960 receives the UI addressed to 
itself, and displays the UI . 

Fig. 105 is a flowchart showing the processing 
performed by the controller 960 when the controller 

15 receives a UI via the wireless LAN, instead of directly 
receiving it from a target model, and displays it. 

The processing performed by the controller 960 
will now be described. At step S1051 the controller 
960 waits until an instruction from a user is input. 

20 At step S1052 a UI request signal for requesting a UI 
and controller identification data for identifying a 
controller are transmitted to the target model by an 
infrared ray communication method. At step S1053 the 
controller 960 waits until a UI signal is output by any 

25 device. At step S1054, the received UI is displayed. 

Fig. 106 is a flowchart showing the processing 
performed by the target model when the controller 




receives a UI via the wireless LAN, instead of directly 
receiving it from a target model, and displays it. The 
processing performed by the controller 960 will now be 
described. 

5 At step S1061 the controller 960 waits until an 

instruction from a user is input. At step S1062 
controller identification data received from the 
controller 960 and the target model identification data 
are transmitted to the server 1041, and issuance of the 

10 UI is requested. 

Fig. 107 is a flowchart showing the processing 
performed by the server 1041 when the controller 
receives a UI via the wireless LAN, instead of directly 
receiving it from a target model, and displays it. The 

15 processing performed by the server 1041 will now be 
described. 

At step S1071 the server 1041 waits until a UI 
transmission request signal is transmitted from the 
target model. At step S1072 the UI of the model, which 
20 is indicated by the target model identification data, 
is transmitted to the controller 960, which is 
indicated by identification data received from the 
target model . 

[ Thirty- seventh Embodiment ] 
25 Fig. 108 is a diagram showing an example where UIs 

of devices are stored in a wireless LAN server 1041 and 
when a controller 960a or 960b faces a facsimile 
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machine 963 as a target: model and requests a UI, a 
different. UI for each user is transmitted, upon a 
request from the facsimile machine 963, from the 
wireless LAN server 1041 to the controller 960a or 
5 960b, and is displayed. 

The controller 960a or 960b, which does not face a 
target device, has a default screen, as is shown in the 
center in Fig. 108. 

The communication between the user, the controller 
10 960a or 960b and the target model will now be 
described. 

(1) A user directs the controller 960a or 960b to a 
target model . 

( 2 ) The user touches a touch panel on the controller 
15 960a or 960b in order to determine the timing at which 

the controller 960a or 960b transmits a UI request 
signal to the target model. Or, the controller 960a or 
960b is set in advance to transmit a UI request signal 
at a predetermined interval . 

20 (3) The controller 960a or 960b transmits controller 
identification data (controller A or controller B) to 
identify itself, user identification data (a senior or 
a beginner) for identifying the user, and a UI request 
signal to the target model. 

25 (4) When the target model receives, from the 

controller 960a or 960b, the controller identification 
data, the user identification data and the UI request 
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signal, the target model transmits, to the server 1041 
that manages the target model, the received controller 
identification data, the user identification data and 
the target model identification data to identify 
5 itself. 

(5) The server 1041 transmits the UI of the target 
model, which is indicated by the user identification 
data and the target model data, via a wireless LAN to 
the controller 960a or 960b, which is indicated by the 

10 controller identification data. In the example in Fig. 

108, a UI that is used for multi-address communication, 
which is difficult for beginners, is transmitted only 
to seniors, not to beginners. 

(6) The controller 960a or 960b receives the UI 
15 addressed to itself, and displays the UI . 

In the above example, the controller transmits the 
level data for a user is transmitted as user 
identification data. However, the controller may 
transmit information, such as the names of users, to 

20 identify individual users, and the target device or the 
server may identify the level of a user from the 
received identification data. An independent UI may be 
prepared for each user. 

The above example has been employed only for the 

25 UI . The contents of urgent mail addressed to the user 
or the schedule to be immediately informed are 
collected in, for example, the server, and they may be 
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included in the UI while the user identification data 
that differs among the users is referred to, and the 
resultant data can be transmitted to the controller. 
Fig. 109 is a flowchart showing the processing 
5 performed by the controller 960 when the controller 

receives a different UI for each user, via the wireless 
LAN, instead of directly receiving it from a target 
model, and displays it. The processing performed by 
the controller 9 60 will now be described. 
10 At step S1091 the controller 960a or 960b waits 

until an instruction from a user is input. At step 

51092 a UI request signal for requesting a UI, 
controller identification data for identifying a 
controller, the user identification data for 

15 identifying the user are transmitted to the target 

model by an infrared ray communication method. At step 

51093 the controller 960a or 960b waits until a UI 
signal is output by any device. At step S1094, the 
received UI is displayed. 

20 Fig. 110 is a flowchart showing the processing 

performed by the target model when the controller 
receives a different UI for each user, via the wireless 
LAN, instead of directly receiving it from a target 
model, and displays it. The processing performed by 

25 the controller 960a or 960b will now be described. 

At step S1101 the controller 960a or 960b waits 
until an instruction from a user is input. At step 
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S1102 the controller identification data received from 
the controller 960, the user identification data, and 
the target model identification data are transmitted to 
the server 1041, and the issuance of the UI is 
5 requested. 

Fig. Ill is a flowchart showing the processing 
performed by the server 1041 when the controller 
receives a different UI for each user via the wireless 
LAN, instead of directly receiving it from a target 

10 model, and displays it. Fig. 112 is a table showing 
UIs that are determined by the target model and user 
identification data. The processing performed by the 
server 1041 will now be described. 

At step Sllll the server 1041 waits until a UI 

15 transmission request signal is transmitted from the 
target model. At step S1112 in accordance with the 
definitions in Fig. 112, the UI that corresponds to the 
target model identification data and the user 
identification data is transmitted to the controller 

20 960a or 960b, which is indicated by identification data 
received from the target model. 
[Thirty-eighth Embodiment] 

Figs. 113 and 114 are diagrams illustrating an 
example where provided is a function that a model to be 

25 operated in front does not have. The communication 
between a user, a controller and a target model will 
now be described. 
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(1) A user faces a controller 960 to a target model, 
and designated a desired function. 

( 2 ) The target model receives a request from the 
controller 960 and transmits it to a server 1041. 

5 (3) The server 1041 transmits a UI of a model that 

corresponds to the requested function. 

(4) The controller 960 displays a received UI . 

( 5 ) The user performs operation according to an 
instruction of the UI . In this example, the user sets 

10 a document to be transmitted to a copier 962 in front, 
and the controller 960 instructs the destination to 
execute the operation. 

( 6 ) The target model handles what it can do and 
requests another apparatus to process what it can not 

15 do. In this case, since the copier 962 reads a 
document but can not facsimile the document, it 
transmits to a facsimile machine 963 the document data 
and the contents of the operation instruction. 

(7) The facsimile machine 963 transmits by fax the 
20 received document data to the instructed destination. 

In the above example, to determine an apparatus 
that the copier 962 asks for the function that the 
copier 9 62 can not perform, each apparatus may have 
knowledge for other apparatuses. Or, the apparatuses 
25 on the network may be queried if they can execute the 

function, and according to the result, the apparatus to 
be asked for may be determined. Or, an inquiry may be 
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issued to the server 1041 that has knowledge of the 
models that correspond to functions. 
[ Thirty-ninth Embodiment ] 

Fig. 115 is a diagram illustrating an example 
5 where, when a controller can not directly communicate 
with a target model in front because an obstacle is 
present in route between the target model and the 
controller, indirect communication is attempted using 
another route . 

10 The communication between the user, the controller 

and the target model will now be described. It should 
be noted that (2) and (3) are not necessarily required. 
When, for example, in (4) a target model generates a 
signal to the controller at predetermined intervals, 

15 the same effect can be obtained. In other words, only 
the essential portion is shown in Fig. 115. 

(1) A user directs the controller 960 to a target 
model . 

( 2 ) The user touches a touch panel on the controller 
20 960 in order to determine the timing at which the 

controller 9 60 transmits a UI request signal to the 
target model. Or, the controller 960 is set in advance 
to transmit a UI request signal at a predetermined 
interval . 

25 (3) The controller 960 transmits a UI request signal 

to the target model . 

(4) A UI signal is transmitted upon the receipt of the 
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UI request signal from the controller 960, or according 
to the setup that a UI request signal at a specified 
interval . 

(5) The controller 960 that receives the UI signal 
5 displays the UI . 

(6) If a UI is not transmitted following a 
predetermined period of time after the UI request 
signal was transmitted, or if a UI is not transmitted 
during a period longer than a predetermined interval, 

10 which is set for transmission of UIs, another route is 
employed to transmit a UI request signal . In this 
example, when infrared communication can not be 
performed, communication through a wireless LAN is 
attempted. 

15 (7) The UI acquired across the LAN is displayed. 

Fig. 116 is a flowchart showing the processing 
performed by a controller for an example where, when 
the controller can not directly communicate with a 
target model in front because an obstacle is present in 

20 route between the target model and the controller, 
indirect communication is attempted using another 
route. The processing performed by the controller 960 
will now be described. 

At step S1161 the controller 960 waits until an 

25 instruction from a user is input. At step S1162 a UI 
request signal for requesting a UI and controller 
identification data for identifying a controller are 
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transmitted to the target model by an infrared ray 
communication method. At step SI 163 a check is 
performed to determine whether or not the UI has been 
received. At step SI 164 a check is performed to 
determine whether a predetermined period of time has 
elapsed following the transmission of the request. At 
step S1165 a UI request signal, for requesting a UI, 
and controller identification data, for identifying a 
controller, are transmitted by a route, such as across 
a wireless LAN, that is different from the route at 
step S1162. At step S1166 the received UI is 
displayed. 

In this case, the UI request signal that is 
directly transmitted to the target model is to be sent 
to the wireless LAN server 1041. However, since the 
wireless LAN server 1041 that receives the UI request 
signal is not sure of a target model, first a menu for 
selecting a model (function) is displayed. A UI 
corresponding to the selected model is displayed. Upon 
receipt of the model selection signal from the 
controller 960, the wireless LAN server 1041 may 
transmit to the controller 960 a UI that corresponds to 
the model, or may transmit the menu and a plurality of 
UIs available from the menu. 
[ Fortieth Embodiment ] 

Fig. 117 is a diagram showing an example where an 
operation and an action performed by a user, the time 
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and associated important: information are stored as 
history. 

An explanation will be given for the processing 
for storing, as history, the operation/action performed 
5 by the user, the time and the associated important 
information . 

(1) A user sets a document in a facsimile machine 963, 
and instructs the transmission of the document and the 
user's name (sender's name) to Mr. a of A company. 

10 (2) According to the instruction, the facsimile 

machine 963 transmits the document to Mr. a. Also 
according to the instruction from Tom, the facsimile 
machine 963 notifies the server 1041 of the fact that 
the document has been transmitted to Mr. a together 

15 with the transmitted document and the transmission 
time. 

(3) The server 1041 saves the received document as a 
file "File ABC" and stores the fact of the receipt as 
history information . 

20 Fig. 118 is a flowchart showing the processing 

performed by the controller for an example where an 
operation and an action performed by a user, the time 
and the associated important information are stored as 
history data. The processing performed by the 

25 controller will now be described. 

At step S1181 the controller 960 waits until an 
instruction from a user is input. At step SI 182 a 




- 144 - 



request signal, for the instruction of the user, 
controller identification data, for identifying a 
controller, and the user identification data, for 
identifying the user, are transmitted by an infrared 
5 ray communication method. At step SI 183 the controller 
960 waits until a UI signal is output by any device. 
At step S1184, the received UI is displayed. 

Fig. 119 is a flowchart showing the processing 
performed by the target model for an example where an 

10 operation and an action performed by a user, the time 
and the associated important information are stored as 
history data. The processing performed by the target 
model will now be described. 

At step S1191 the target model waits until a 

15 request signal is transmitted by the controller 960. 

At step SI 192 a process corresponding to the request 
signal from the controller 960 is performed. In the 
example shown in Fig. 117 a document set by the user is 
read and transmitted to Mr. a of A company. At step 

20 S1193 controller identification data received from the 
controller 960, the user identification data, and 
target model identification data, and important data 
associated with the action that was performed, are 
transmitted to the server 1401. In the example in Fig. 

25 117, according to the instruction from Tom, the fact 

that the document was transmitted to Mr. a is sent to 
the server 1041, together with the transmission time 
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and the transmitted document that is the important 
information associated with the fact. 

Fig. 120 is a flowchart showing the processing 
performed by the server for an example where an 
5 operation and an action performed by a user, the time 
and the associated important information are stored as 
history data. Fig. 121 is a diagram showing example 
history information that is updated as the result of 
processing. The processing performed by the server 
-if 10 will now be described. 

^ At step S1201 the server waits until it receives a 

Fhj 

H request signal from the target model. At step S1202 

UJ the important information associated with the received 

M fact is stored. In the example in Fig. 117 the 

: y 

fy 15 document the transmission of which is instructed by Tom 

%b is saved as File ABC. 

At step S1203 the history data are updated in 
consonance with the received fact. In the example 
shown in Fig. 117 the fact that Tom transmitted a 
20 document to M. a of A company at 10:00 on July 5, 1996 
is recorded together with the file name stored at step 
S1201. Fig. 121 is a table showing updated history 
data; the fact that Mary printed ten copies of File XYZ 
on July 3, 1996, and the fact that Tom transmitted File 
25 ABC to Mr. a of A company at 10:00 on July 5, 1996. At 
step S1204 a UI that indicates the normal termination 
of the processing is transmitted to the controller that 
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is defined by the controller identification data 
received from the target model, 
[ Forty- first Embodiment ] 

Figs, 122 and 123 are diagrams showing an example 
5 where an operation/action performed by a user, the time 
and associated important information are stored as 
history data and based on the history data an operation 
is performed. 

aaa An explanation will be described the processing 

^ 10 for the example where an operation/action performed by 

%3 a user, the time and associated important information 

O are stored as history data and based on the history 

UJ data an operation is performed. 

H (1) A controller 960 instructs a copier 962 to show 

Hi 

fU 15 the history of Tom. 

(2) Upon receipt of the instruction, the copier 962 
asks a server 1401 for managing the history to show the 
history of Tom. 

(3) The server 1041 refers to the history data and 
20 transmits, as a UI, only the history associated with 

the Tom to the copier 962. The copier 962 transmits 
the received history data to the controller 960. 

(4) Tom selects a specific fact from the history data 
displayed on the controller 960, and instructs an 

25 action. In this case, since Tom needs a material that 
was previously sent to A company, the fact "File ABC 
was sent to Mr. a of A company" is selected and action 
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"copy" is instructed "to acquire the material from the 
copier 962 in front. 

(5) The copier 962 transmits to the server 1041 the 
instruction "copy File ABC" from the user. 
5 ( 6 ) The server 1041 sends a corresponding document 
"File ABC" to the copier 962. 

(7) The copier 962 prints a document received by the 
server 1041. 

In the above example, the fact "File ABC was sent 
10 to Mr. a of A company" is selected from the history in 
order to acquire necessary material from the copier 962 
in front. When the name of document "File ABC" to be 
output is obtained, the list of files can be displayed 
instead of the history data and the file name can be 
15 selected, or the file name can be input directly. 

In addition, in the above example, the document to 
be extracted is the one that is transmitted by the 
facsimile machine 963 and stored in the server 1041. A 
document stored in a PC across the network can also be 
20 employed. 

Fig. 124 is a diagram showing the structure of the 
controller 960 used in this embodiment. 

A display 1241 is used to display data on a liquid 
crystal screen. A resistance film is attached to the 
25 display 1241, and data can be input with a pen 1242. A 
button on a screen may be selected by the pen 1242. 
Furthermore, a character recognition function is 
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provided, and the characters written with the pen 1242 
can be input. Instead of this function, a voice 
input/recognition function may be employed to input 
information. 

The input information can be transmitted from a 
communication unit 1243 to another apparatus. A CPU 
1244 for controlling and a storage unit 1245 are 
provided inside the apparatus, and input information 
and information received by the communication unit 1243 
and the history of the operations are stored. The 
storage unit 1245 can be a SRAM card, an SIMM or an 
HDD. The controller 960 is battery operated and water 
resistant . 

Fig. 125 is a diagram illustrating another example 
structure of the controller 960 used in the embodiment - 

A display 1251 is used to display data using 
liquid crystal or a CRT. A touch panel can be 
additionally provided as an option to the display 1251. 
Thus, information can be input by a finger or a pen. 
An input unit 1252 is used to input information by 
manipulation of a button. The input data can be 
transmitted from a communication unit 1253 to another 
apparatus. A CPU 1254 for controlling and a storage 
unit 1255 are provided inside the apparatus, and input 
information and information received by the 
communication unit 1253 and the history of the 
operations are stored. The storage unit 1255 can be a 
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SRAM card, an SIMM or an HDD. 

The controller in Fig. 125 can be used as a button 
controller with the storage unit 1255 being removed. 
The controller can be used as a personal digital 
5 assistant (PC) with the communication unit 1253 being 

removed. In addition, the controller can be used as is 
shown in Fig. 124, with the input unit 1252 being 
removed and with the touch panel provided. 

Fig. 126 is a diagram showing an additional 
10 example structure of the controller 960 used in this 
embodiment . 

A display 1261 is used to display data using 
liquid crystal. An input unit 1262 is used to input 
information by manipulating a button. The input data 

15 can be transmitted by an infrared communication unit 

1263 to another apparatus. A CPU 1264 for controlling 
and a storage unit 1265 are provided inside the 
apparatus, and input information and information 
received by the infrared communication unit 1263 and 

20 the history of the operations are stored. The storage 
unit 1265 can be a SRAM card, an SIMM or an HDD. 
Furthermore, a PCMCIA slot 1266 is provided, and, for 
example, a PCMCIA card is loaded into the slot 1266 to 
enable the connection by a PCMCIA bus, so that two 

25 communication channels can be employed. The infrared 
communication unit 1263 may be used only for 
transmission by light emission, and the PCMCIA bus may 
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be used for reception. Of course, the connection by 
the PCMCIA bus may not be employed. 
[ Forty- second Embodiment ] 

This system arrangement according to this 
5 embodiment is as shown in Figs. 41A and 41B, and 

information processing apparatuses 411, 412 and 413 can 
communicate with each other. 

When the information processing apparatus 411 
needs information stored in the information processing 
10 apparatus 412, or when the information is to be 

transmitted to the information processing apparatus, or 
when an instruction is to be transmitted to the 
information processing apparatus 412, the information 
processing apparatus 411 requests the communication 
15 with the information processing apparatus 412 and 
exchanges required information and an instruction. 

Any apparatus can be used for the information 
processing apparatuses 411, 412 or 413 so long as they 
have necessary communication function; a printer, a 
20 scanner, a facsimile machine, a desktop PC or a digital 
camera can be used. 

Fig. 127 is a diagram illustrating the functional 
arrangement for this embodiment. 

An operating unit 41 displays a screen required 
25 for a user to operate in accordance with data stored in 
an operation data memory area 42 . An operation 
performed by a user or an instruction issued by a user 
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is acquired by the operation unit 41 and transmitted to 
a task reception unit 43. 

The operation data 42 is data to be referred to by 
the operation unit 41, and specifies the process at the 
5 operation unit 41. In this embodiment, based on the 

information received by the task reception unit 43, the 
operation data 42 is altered or updated by a process 
execution unit 47, which will be described later, so 
m that flexible procedures can be provided, 

"ij 10 One of methods for operating an apparatus other 

r£ than the information processing unit 411, i.e., the 

W information processing apparatus 412, is the one for 

W acquiring the operation data 42 for the information 

M' processing apparatus 412 and for altering and updating 

RJ 15 the operation data 42 of the information processing 

J3 apparatus 411. 

In another example, the presence of the 
information processing apparatus 412, which is 
connected across the network to the information 
20 processing apparatus 411 in this embodiment, is 

detected, so that a new function obtained by combining 
the two information processing apparatuses can be 
implemented. Information required for an instruction 
for the function is added to the operation data 42, and 
25 thus, the function obtained by combining a plurality of 
apparatuses can be provided. 

The task reception unit 43 receives not only 
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information from the operation unit 41 , but also 
receives, as the status change, a change in the 
environment of the information processing apparatus 411 
or a task received from another apparatus, such as the 
5 information processing apparatus 412. Task reception 
unit 43 also detects the connection of a new 
information processing apparatus to the network, 
receives as a task a print instruction from another 
apparatus, or detects that no process is performed 
10 during a specified period of time. As a result, 

received information is added as a new task to a task 
table 44. 

The task reception unit 43 is required as well as 
the process execution unit 47, which will be described 
15 later, to communicate with different apparatuses. 

The task table 44 is a table in which are stored 
tasks having the activation condition and the contents . 
Figs. 129A to 129F are examples for the task table 44. 
In Figs. 129A to 129F, the tasks are stored from 
20 the top of the table in the order of priorities. The 

addition of a task to the task table 44 is performed by 
the previously mentioned task reception unit 43 and the 
process execution unit 47 to be described later. 
Extraction of the task from the task table 44 is 
25 performed by a task analysis unit 45, which will be 
described later. 

The task analysis unit 45 extracts a task having 
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the highest priority from the task table 44, and 
analyzes the object of the task. Since the task 
analysis unit 45 analyzes the object, a planning 
decision unit 46 and the process execution unit 47 can 
avoid inefficient processing and can perform an optimal 
process . 

For example, a task 63 in the task table 44 in 
Figs. 129A to 129F is analyzed and understood that the 
direct object is printing and the final object is to 
transmit a printed document to a user. If the final 
object can be achieved by an efficient method, it is 
found that even printing is unnecessary, and planning 
is performed according to that effect. The information 
to be processed is also analyzed, and it is found that 
format conversion is required. 

The planning decision unit 46 receives the 
analysis from the task analysis unit 45, makes planning 
for a process to be performed. For example, for the 
task 63 stored in the task table 44 in Figs. 129A to 
129F, a process for extracting an object from an 
apparatus B before printing of the object is planned. 

The process execution unit 47 performs a process 
according to the contents determined by the planning 
decision unit 46, e.g., sends an instruction to another 
apparatus. The process execution unit 47 will be 
described in detail in a forty-third embodiment and 
following embodiments. The functional arrangement in 
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Fig. 127 can be provided by -the hardware configuration 
in Fig. 3. 

Fig. 128 is a flowchart: showing the processing for 
this embodiment:. 
5 When the information processing apparatus 411 in 

this embodiment is activated, at step S51 the task 
table 44 is initialized as is denoted by 61 in Fig. 
129A. 

Then, at step S52 a check is performed to 
10 determine whether the task reception unit 43 has 

received, as the status change, information that is 
acquired by the operation unit 41 for an operation 
instruction from a user, an environmental change, or a 
task from another apparatus. When the task is 
15 received, program control advances to step S53, and as 
is shown in Fig. 129B, the analysis task 62 of the 
received information is added to the head of the task 
table 44. 

Following this, at step S54 a check is performed 
20 to determine whether or not an executable task is 
present in the task table 44. If there is no such 
task, program control returns to step S52, whereat the 
reception of the task is examined again. If there is 
such a task, program control moves to step S55, whereat 
25 the task having the highest priority is extracted from 
the task table 44. 

At step S56 the object of the task is analyzed, 
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and at step S57 the process to be performed is planned 
and decided. If the task table is in the state shown 
in Fig. 129B, highest priority task 62 "ANALYZE" is 
extracted, it is apprehended that its object is 
5 analysis of received information, and planning for 
analysis is performed. 

At step S58 the process according to the planning 
is performed, and as needed, a new task is added to the 
task table 44. When, for example, the received 

10 information is analyzed according to the plan obtained 
from the task 62 "ANALYZE" in Fig. 129B, the contents 
"Print 1 contract . doc ' from B" which is instructed by a 
user through the operation unit 41, are analyzed and a 
new task 63 "PRINT" is added as is shown in Fig. 129C. 

15 Program control returns again to step S52. 

However, since no new task is received, program control 
moves to step S54, and the above process is repeated. 
As a result, task 63 "PRINT" in Fig. 129C, which was 
added at step S58, is extracted, analyzed and planned. 

20 To execute task PRINT, it is found that the object must 
be present in the information processing apparatus 411, 
and new task 64 "PULL" and task 65 "PRINT" which 
employs the "pulled" object, are added as is shown in 
Fig. 129D. 

25 In addition, when the processing is repeated, the 

task 64 "PULL" in Fig. 129D is executed. Since the 
contents of the task PULL are important portions for 
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the present invention, detailed explanation for them 
will be given for the forty-third embodiment and the 
following. 

When the process is further repeated, the object 
5 that is acquired as the result of the task PULL is 

employed to perform task 65 in Fig. 129E. As a result, 

the task table 44 is returned to the initial state, as 

is shown in Fig. 129F. 

[ Forty- third Embodiment ] 
10 Fig. 130 is a diagram showing an example 

input/output device that can operate another apparatus. 
The operation procedure (UI) implemented by the 

operation unit 41 is performed by referring to the 

operation data 42 that the information processing 
15 apparatus 411 originally includes, or by referring to 

the operation data 42 that is acquired from another 

information processing apparatus 412 connected to the 

network . 

An instruction 71 by a user's voice, or the 
20 employment of an operation panel screen 72 to designate 
a target device, is performed by referring to the 
operation data 42 that are originally included in a 
copier, which serves as the information processing 
apparatus 411. 

25 The copier 74, which is the information processing 

apparatus 411, executes the task PULL at the process 
execution unit 47, and refers to on the operation data 
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42, which are acquired from a facsimile machine 75 that: 
serves as the information processing apparatus 412 
connected via the network, to designate a transfer 
destination and a condition by using the screen 73 of 
5 the operation panel. 

The operation data 42 of the facsimile machine 75 
may be held by the copier 74 in advance, and be 
referred to as needed. 
E3 The operation data 42 of the facsimile machine 75 

%U 10 may be acquired from another device, such as a server. 

CO A specific method for communication between the 

yj information processing apparatuses 411 and 412 will be 

? explained in detail for a forty-ninth embodiment and 

ry the following embodiments. 

ry 

_q 15 [Forty- fourth Embodiment] 

.11 Fig. 131 is a diagram showing an input/output 

apparatus that can operate a function obtained by a 
combination of a plurality of devices. 

The procedure (UI: User Interface) implemented by 
20 the operation unit 41 is performed by referring to the 
operation data 42 the information processing apparatus 
411 originally includes, or by referring to the 
operation data 42 that is updated as the result of the 
detection of the information processing apparatus 412 
25 connected via the network. 

The number of copies using the operation panel 81 
in Fig. 131 is designated by referring to the operation 
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data 42 that is originally included in the copier 74, 
which serves as the information processing apparatus 
411. The "Copy" displayed in the lower portion of the 
operation panel indicates an instruction of copying, 
and "transmit" displayed in the hatched state indicates 
that the transmission process is disabled. 

As the result of the detection of the facsimile 
machine 75, which serves as the information processing 
apparatus 412 connected via the network, the operation 
data 42 of the copier 74 are updated to the function 
that can be used, i.e., the transmission function from 
which the hatching is removed, and is referred to, so 
that the number of copies on a display 82 is designated 
using the operation panel, as well as the operation 
panel 81. 

When a user selects and instructs the execution of 
the transmission function, a display 83 for designating 
the transmission destination is provided and the 
transmission function can be executed. As is described 
above, when a user need only set a desired document on 
a copier near the user and instruct the execution of 
the transmission function, the document can be 
transmitted by using the function of the facsimile 
machine 75, which is at a remote area and is connected 
via a network. This accomplishes both the document 
input function of the copier and the document 
transmission function of the facsimile machine. 
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When the facsimile machine 75 is not connected on 
the display 81, the transmission operation on the 
display 83 for designating the transmission destination 
is performed in advance, and at the same time that the 
connection of the facsimile machine 75 is detected, the 
contents of the transmission operation is transmitted 
to the facsimile machine 75. Thus, in consonance with 
the contents, the automatic transmission can be 
performed, or the reservation operation, such as a 
confirmation to a user before automatic transmission, 
as is shown in a screen 84, can be performed. 

In this embodiment, while the facsimile machine 75 
is not connected, the display "transmit" for a 
transmission instruction is displayed by hatching to 
indicate transmission is enabled. Such a display may 
not be provided at all. 

A specific method for communication between the 
information processing apparatuses 411 and 412 will be 
described in detail for the forty-ninth embodiment and 
the following. 
[ Forty- f i f th Embod iment ] 

Fig. 132 is a diagram illustrating an example 
input /output device that can communicate with a user. 

In the example in Fig. 132, a user sends an 
instruction 93 to a copier 91 that is physically 
located in a remote area in order to permit a MyDesk 
92, a desktop personal computer, to display the current 
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location of the user. 

The instruction 93 "Display present place on 
<MyDesk>", which is sent to the copier 91 that serves 
as the information processing apparatus 411, is 
5 transmitted from the operation unit 41 to the task 
reception unit 43 . The contents are analyzed and 
planning is performed on the results. Then, the 
execution unit 47 executes the final data as an 
instruction task INDICATE for a PC 92, which serves as 
10 the information processing apparatus 412 connected via 
the network . 

The PC 92, which is the information processing 
apparatus 412 connected via the network, receives 
communication task COMMUNICATION from the reception 
15 unit 43, analyzes a received instruction, and perform 
planning on the results. Then, the execution unit 47 
displays a message on a screen. 

A specific method for communication between the 
information processing apparatuses 411 and 412 will be 
20 described in detail for the forty-ninth embodiment and 
the following. 
[ Forty- s ixth Embodiment ] 

Fig. 133 is a diagram illustrating an example 
input/output device that can acquire information from 
25 another apparatus. 

In the example in Fig. 133, a user sends an 
instruction 1333 to a printer 1331 that is located in a 
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physically remote area "to print information 1334 
"contract . doc" that is stored in apparatus B1332. 

The instruction 1333 "Please fetch ' contract . doc ' 
from B and print it" which the user sent to the printer 
5 1331 that serves as the information processing 

apparatus 411, is transmitted from the operation unit 
41 to the task reception unit 43. The contents are 
analyzed and planning is performed on the results. 
Then, the execution unit 47 executes the final data as 
10 information acquisition task PULL for the apparatus B 
1332, which serves as the information processing 
apparatus 412 connected via the network. The apparatus 
B can be a printer, a scanner, a desktop PC or a filing 
server . 

15 The apparatus B1332, which serves as the 

information processing apparatus 412 connected via the 
network, receives the information acquisition task PULL 
from the reception unit 43, analyzes the received 
instruction, and performs planning on the data. The 

20 execution unit 47 then transmits corresponding 
information 1334 "contract.doc". 

A specific method for communication between the 
information processing apparatuses 411 and 412 will be 
described in detail for the forty-ninth embodiment and 

25 the following. 

[Forty- seventh Embodiment] 

Fig. 134 is a diagram illustrating an example 
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input/output: device that can be stored in another 
apparatus . 

In the example shown in Fig. 134, a user reads a 
document 115 using a scanner 111 and sends an 
5 instruction 113 to permit a filing server 112 at a 

physically remote area to perform filing with a title 
and an index . 

The instruction 113 "Filing to Filing Server, 
Title, Index Extraction" which the user sent to the 

10 scanner 111 that serves as the information processing 
apparatus 411, is transmitted from the operation unit 
41 to the task reception unit 43 . The contents are 
analyzed and planning is performed on the results. 
Then, the execution unit 47 executes the final data as 

15 information storage task PUSH for the filing server 
112, which serves as the information processing 
apparatus 412 connected via the network. 

The filing server 112, which serves as the 
information processing apparatus 412 connected via the 

20 network, receives the information storage task PUSH 
from the reception unit 43, analyzes the received 
instruction, and performs planning on the data. The 
execution unit 47 then files corresponding information 
115 " contract . doc " - 

25 A specific method for communication between the 

information processing apparatuses 411 and 412 will be 
described in detail for the forty-ninth embodiment and 
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■the following. 

[Forty- eighth Embodiment:] 

Fig. 135 is a diagram illustrating an example 
input/output device that can be stored in another 
5 apparatus. 

In the example shown in Fig. 135, a user sends an 
instruction 123 to a printer 121 to transmit 
information 125, which is stored in a filing server 122 
located at a physically remote area, to John 124 who is 

10 located at a physically farther area. 

The instruction 123 "Action: SEND, What: abc.doc. 
From: Filing Server, To: John" which the user sent to 
the printer 121 that serves as the information 
processing apparatus 411, is transmitted from the 

15 operation unit 41 to the task reception unit 43. The 
contents are analyzed and planning is performed on the 
results. Then, the execution unit 47 executes the 
final data as an instruction task for the filing server 
122, which serves as the information processing 

20 apparatus 412 connected via the network. 

The filing server 122, which serves as the 
information processing apparatus 412 connected via the 
network, receives the instruction task from the 
reception unit 43, analyzes the received instruction, 

25 and performs planning on the data. The execution unit 
47 then transmits corresponding information 125 
"abc.doc" to John by E-mail. 
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A specific method for communication between the 
information processing apparatuses 411 and 412 will be 
described in detail for the forty-ninth embodiment and 
the following. 
5 [Forty-ninth Embodiment] 

Figs, 136A and 136B are flowcharts showing the 
processing for <COMMUNICATE> task. 

<COMMUNICATE> task is performed when arbitrary 
apparatuses A and B need communicate with each other. 
10 In Figs. 136A and 136B are shown the processing that 

the apparatus A ( a source ) that transmits a COMMUNICATE 
request is to execute for communication with the 
apparatus B (destination) that receives a COMMUNICATE 
request . 

15 At step S1312, the apparatus A contacts the 

apparatus B to establish the connection for 
communication. At step S1313 a check is performed to 
determine where a reply is received from the apparatus 
B. If there is no response from the apparatus B, at 

20 step S1314 a check is performed to determine whether or 
not a user or another person should be notified of no 
response. If notification is necessary, at step S1315 
a notification is issued to a user by an appropriate 
means/medium. At step S1316, according to the 

25 decision/assumption of the system or the setup by the 
user, a check is performed to determine whether the 
task should be re-execute by the system later. If the 
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re-execution of the task is necessary, at step S1317 
the task is added to the task table, and the processing 
is thereafter terminated. If the re-execution of the 
task is not required, at step S1318 the task is 
5 deleted, and is added to "failed list". The processing 
is thereafter terminated. 

When, at step S1313, a response is received, at 
step S1319 whether the response is "Will call back" is 
determined. If the response is "Will call back" the 

10 apparatus B will call back the apparatus A later. At 
step S1320, a check is performed to determine whether 
or not the user or another person should be notified of 
the response. If notification is necessary, at step 
S1321 the notification is issued to the user by an 

15 appropriate means/medium. At step S1322 the system 

adds the task to the task table, and remembers that the 
apparatus B will call back so as to cope with the call 
back. The processing is thereafter terminated. 

If, at step S1319, the response is not "Will call 

20 back" at step S1323 whether the response is "Contact 
later" is determined. If the response is "Contact 
later" it is assumed that the apparatus B wants a 
contact later, and the above described processes at 
step S1314 and the following steps are performed. 

25 When, at step S1323, the response is not "Contact 

later" it is assumed that the response is "OK" and at 
step S1324 the response is confirmed. If the response 
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is not "OK" an error occurs. At step S1331 an error 
message is displayed. 

When the response is "OK" at step S1325 a check is 
performed to determine whether or not a negotiation 
with the apparatus B is required for the current 
operation. If the negotiation is required, at step 
S1326 a negotiation is initiated. The object to be 
negotiated includes a data format for data execute and 
protocol/version exchange, and a data format for other 
information required in advance for correct 
communication. This negotiation is repeated until the 

communication base used in common for the apparatuses A 

> 

and B is established. 

At step S1327 information to be transmitted is 
issued as an request from the apparatus A to the 
apparatus B. At step S1328, whether or not a response 
is received. When the response is received, at step 
S1329 a check is performed to determine whether there 
is an action to be executed relative to the received 
response. The action is, for example, conversion of a 
format or erasure of an unnecessary header. The object 
of the communication includes actions such as printing, 
transmission, filing, voice output and display. This 
means that necessary actions should be executed, and at 
step S1330 these actions are added to the task table. 

When, at step S1328, there is no response, the 
above described processes at step S1314 and the 
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following steps are performed. The above processing 
loop is continued until requested information is 
obtained or until the system can not continue the 
processing . 
[ Fi f t ieth Embodiment ] 

Figs. 137A and 137B are flowcharts showing the 
processing for <PULL> task. 

<PULL> task is performed when an arbitrary 
apparatus A needs to extract information from an 
apparatus B. In Figs. 137A and 137B are shown the 
processing that the apparatus A (a source) that 
transmits a PULL request is to execute for 
communication with the apparatus B (destination) that 
receives a PULL request. 

At step S1412, the apparatus A contacts the 
apparatus B to establish the connection for 
communication. At step S1413 a check is performed to 
determine where a reply is received from the apparatus 
B. If there is no response from the apparatus B, at 
step S1414 a check is performed to determine whether or 
not a user or another person should be notified of no 
response. If notification is necessary, at step S1415 
a notification is issued to a user by an appropriate 
means/medium. At step S1416, according to the 
decision/assumption of the system or the setup by the 
user, a check is performed to determine whether the 
task should be re-execute by the system later. If the 
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re-execution of the "task is necessary, at step S1417 
the task is added to the task table, and the processing 
is thereafter terminated. If the re-execution of the 
task is not required, at step S1418 the task is 
5 deleted, and is added to "failed list". The processing 
is thereafter terminated. 

When, at step S1413, a response is received, at 
step S1419 whether the response is "Will call back" is 
determined. If the response is "Will call back" the 

10 apparatus B will call back the apparatus A later. At 
step S1420, a check is performed to determine whether 
or not the user or another person should be notified of 
the response. If notification is necessary, at step 
S1421 the notification is issued to the user by an 

15 appropriate means/medium. At step S1422 the system 

adds the task to the task table, and remembers that the 
apparatus B will call back so as to cope with the call 
back. The processing is thereafter terminated. 

If, at step S1419, the response is not "Will call 

20 back" at step S1423 whether the response is "Contact 
later" is determined. If the response is "Contact 
later" it is assumed that the apparatus B wants a 
contact later, and the above described processes at 
step S1414 and the following steps are performed. 

25 When, at step S1423, the response is not "Contact 

later" it is assumed that the response is "OK" and at 
step S1424 the response is confirmed. If the response 



- 169 - 



is not "OK" an error occurs. At step S1431 an error 
message is displayed. 

When the response is "OK" at step S1425 a check is 
performed to determine whether or not a negotiation 
5 with the apparatus B is required for the current 

operation. If the negotiation is required, at step 
S1426 a negotiation is initiated. The object to be 
negotiated includes a data format for data execute and 
protocol/version exchange, and a data format for other 

10 information required in advance for correct 

communication. This negotiation is repeated until the 
communication base used in common for the apparatuses A 
and B is established. 

At step S1427 information to be extracted is 

15 issued as a request from the apparatus A to the 

apparatus B. At step S1428, whether or not requested 
information is received. When the requested 
information is received, at step S1429 a check is 
performed to determine whether there is an action to be 

20 executed relative to the received response. The action 
is, for example, conversion of a format or erasure of 
an unnecessary header. The object of the extraction of 
information includes actions such as printing, 
transmission, filing, voice output and display. This 

25 means that necessary actions should be executed, and at 
step S1430 these actions are added to the task table. 

When, at step S1428, requested information is not 




acquired, the above described processes at step S1414 
and the following steps are performed. The above 
processing loop is continued until requested 
information is obtained or until the system can not 
5 continue the processing. 
[Fifty- first Embodiment] 

Figs. 138A and 138B are flowcharts showing the 
processing for <PUSH> task. 

<PUSH> task is performed when an arbitrary 

10 apparatus A need to transmit information to an 

apparatus B. In Figs. 138A and 138B are shown the 
processing that the apparatus A (a source) that 
transmits a PUSH request is to execute for 
communication with the apparatus B (destination) that 

15 receives a PUSH request. 

At step S1512, the apparatus A contacts the 
apparatus B to establish the connection for 
communication. At step S1513 a check is performed to 
determine where a reply is received from the apparatus 

20 B. If there is no response from the apparatus B, at 

step S1514 a check is performed to determine whether or 
not a user or another person should be notified of no 
response. If notification is necessary, at step S1515 
a notification is issued to a user by an appropriate 

25 means/medium. At step S1516, according to the 

decision/assumption of the system or the setup by the 
user, a check is performed to determine whether the 
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task should be re-execute by the system later. If the 
re-execution of the task is necessary, at step S1517 
the task is added to the task table, and the processing 
is thereafter terminated. If the re-execution of the 
5 task is not required, at step S1518 the task is 

deleted, and is added to "failed list". The processing 
is thereafter terminated. 

When, at step S1513, a response is received, at 
step S1519 whether the response is "Will call back" is 

10 determined. If the response is "Will call back" the 
apparatus B will call back the apparatus A later. At 
step S1520, a check is performed to determine whether 
or not the user or another person should be notified of 
the response. If notification is necessary, at step 

15 S1521 the notification is issued to the user by an 
appropriate means/medium. At step S1522 the system 
adds the task to the task table, and remembers that the 
apparatus B will call back so as to cope with the call 
back. The processing is thereafter terminated. 

20 If, at step S1519, the response is not "Will call 

back" at step S1523 whether the response is "Contact 
later" is determined. If the response is "Contact 
later" it is assumed that the apparatus B wants a 
contact later, and the above described processes at 

25 step S1514 and the following steps are performed. 

When, at step S1523, the response is not "Contact 
later" it is assumed that the response is "OK" and at 
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step S1524 the response is confirmed. If the response 
is not "OK" an error occurs. At step S1531 an error 
message is displayed. 

When the response is 'OK" at step S1525 a check is 
performed to determine whether or not a negotiation 
with the apparatus B is required for the current 
operation. If the negotiation is required, at step 
S1526 a negotiation is initiated. The object to be 
negotiated includes a data format for data execute and 
protocol /version exchange, and a data format for other 
information required in advance for correct 
communication. This negotiation is repeated until the 
communication base used in common for the apparatuses A 
and B is established. 

At step SI 527 information to be transmitted is 
issued as a request from the apparatus A to the 
apparatus B. At step SI 528, whether or not a response 
is received. When the response is received, at step 
SI 529 a check is performed to determine whether there 
is an action to be executed relative to the received 
response. The action is, for example, conversion of a 
format or erasure of an unnecessary header. The object 
of the communication includes actions such as printing, 
transmission, filing, voice output and display. This 
means that necessary actions should be executed, and at 
step SI 530 these actions are added to the task table. 

When, at step S1528, there is no response, the 
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above described processes at step SI 514 and the 
following steps are performed. The above processing 
loop is continued until requested information is 
obtained or until the system can not continue the 
5 processing. 

[ Fi f ty- second Embodiment ] 

Figs. 139A and 139B are flowcharts showing the 
processing for <INDICATE> task. 

<INDICATE> task is performed when an arbitrary 

10 apparatus A needs to send an instruction to an 

apparatus B. In Figs. 139A and 139B are shown the 
processing that the apparatus A (a source) that 
transmits an INDICATE request is to execute for 
communication with the apparatus B (destination) that 

15 receives an INDICATE request. 

At step SI 61 2, the apparatus A contacts the 
apparatus B to establish the connection for 
communication. At step S1613 a check is performed to 
determine where a reply is received from the apparatus 

20 B. If there is no response from the apparatus B, at 

step S1614 a check is performed to determine whether or 
not a user or another person should be notified of no 
response. If notification is necessary, at step S1615 
a notification is issued to a user by an appropriate 

25 means/medium. At step S1616, according to the 

decision/assumption of the system or the setup by the 
user, a check is performed to determine whether the 
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•task should be re-execute by the system later. If the 
re-execution of the task is necessary, at step S1617 
the task is added to the task table, and the processing 
is thereafter terminated. If the re-execution of the 
5 task is not required, at step S1618 the task is 

deleted, and is added to "failed list". The processing 
is thereafter terminated. 

When, at step SI 613, a response is received, at 
step S1619 whether the response is "Will call back" is 

10 determined. If the response is "Will call back" the 
apparatus B will call back the apparatus A later. At 
step S1620, a check is performed to determine whether 
or not the user or another person should be notified of 
the response. If notification is necessary, at step 

15 S1621 the notification is issued to the user by an 
appropriate means/medium. At step S1622 the system 
adds the task to the task table, and remembers that the 
apparatus B will call back so as to cope with the call 
back. The processing is thereafter terminated. 

20 If, at step S1619, the response is not "Will call 

back" at step SI 623 whether the response is "Contact 
later" is determined. If the response is "Contact 
later" it is assumed that the apparatus B wants a 
contact later, and the above described processes at 

25 step SI 614 and the following steps are performed. 

When, at step S1623, the response is not "Contact 
later" it is assumed that the response is "OK" and at 
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step SI 624 the response is confirmed. If the response 
is not "OK" an error occurs. At step SI 631 an error 
message is displayed. 

When the response is f 0K" at step S1625 a check is 
5 performed to determine whether or not a negotiation 
with the apparatus B is required for the current 
operation. If the negotiation is required, at step 
SI 626 a negotiation is initiated. The object to be 
negotiated includes a data format for data execute and 

10 protocol/version exchange, and a data format for other 
information required in advance for correct 
communication. This negotiation is repeated until the 
communication base used in common for the apparatuses A 
and B is established. 

15 At step SI 627 according to the operation 

instruction by the user employing the operation unit 
41, an instruction is issued as an request from the 
apparatus A to the apparatus B. At step SI 628, a check 
is performed to determine whether or not a response is 

20 received. When the response is received, at step SI 629 
a check is performed to determine whether there is an 
action to be executed relative to the received 
response. The action is, for example, conversion of a 
format or erasure of an unnecessary header. The object 

25 of the communication includes actions such as printing, 
transmission, filing, voice output and display. This 
means that necessary actions should be executed, and at 
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step SI 630 these actions are added to the task table. 

When, at step SI 6 28, there is no response, the 
above described processes at step S1614 and the 
following steps are performed. The above processing 
5 loop is continued until requested information is 
obtained or until the system can not continue the 
processing. 

[Fifty-third Embodiment] 

Figs. 140A and 140B are flowcharts showing the 

10 processing for <RESPOND> task. 

<RESP0ND> task is performed when an apparatus B 
(destination) that receives a COMMUNICATE request sends 
a response. In Figs. 140A and 140B are shown the 
processing that is to be executed by the apparatus B 

15 that receives a COMMUNICATE request (e.g., PULL) from a 
transmission apparatus (source) A. 

At step S1712 a check is performed to determine 
whether or not a request type is contact. If the 
request is contact, the apparatus A attempts to 

20 establish connection with the apparatus B for 

communication. At step S1713 the current status or a 
specific user's set value is allocated for status SI. 
At step S1314 whether the status SI is a "Very busy" 
mode or a "Standby 11 mode is determined. If the status 

25 SI is either mode, it is assumed that the apparatus B 

can not accept the request from the apparatus A. Then, 
at step S1715 the optimal response is decided. A check 
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is performed to determine whether or not the apparatus 
B can call back the apparatus A any time. If the 
apparatus B can call back, program control moves to 
step S1716 and response "Will call back" is 
5 transmitted. If the apparatus B can not call back, at 
step S1717 response "Contact later" is transmitted. 
This means that the apparatus A must contact the 
apparatus B later. When, at step S1714, the status Al 
is neither a "Very busy" mode nor a "Standby" mode, 

10 response "OK" is transmitted. The processing is 
thereafter terminated. 

When the request type is not contact, at step 
S1719 a check is performed to determine whether the 
request type is a request for information. If so, at 

15 step S1720 a response is transmitted relative to the 

request for information, which will be described later 
in Fig. 141. When the request type is not a request 
for information, at step S1721 a check is performed to 
determine whether the request type is a request for 

20 storage of information. If so, at step S1722 a 
response relative to the request for storage of 
information, which will be described later in Fig. 142, 
is issued. If the request type is not a request for 
storage of information, at step S1723 a check is 

25 performed to determine whether the request type is a 
request for an instruction. If so, at step S1724 a 
response relative to the request for an instruction, 
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which will be described later, is issued. If the 
request type is not a request for an instruction, at 
step S1725 an optimal response is issued, and the 
processing is thereafter terminated. 
5 Fig. 141 is a flowchart showing a response task 

relative to the request for information. In this 
flowchart, a process will be explained that is 
initiated when the request type is a request for 
information (step S1720 in Figs. 140A and 140B). 

10 At step S1812 a check is performed to determine 

whether or not negotiation is required with the 
apparatus A that issued the request for an instruction. 
If the negotiation is required, program control moves 
to step S1813, whereat the negotiation is initiated. 

15 The object to be negotiated includes a data format for 
data execute and protocol/ version exchange, and a data 
format for other information required in advance for 
correct communication. This negotiation is repeated 
until the communication base used in common for the 

20 apparatuses A and B is established. 

Following this, at step S1814 fetching of 
requested information is performed. At step S1815, a 
check is performed to determine whether or not the 
requested information is found. If the information is 

25 not found, at step S1816 response = NG is transmitted 
and the processing is terminated. 

If the requested information is found, at step 
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S1817 a check is performed to determine whether the 
information should be transferred immediately. If the 
immediate transmission is required, program control 
moves to step S1820, whereat a check is performed to 
5 determine whether or not data formatting or data 

conversion is required. If data formatting or data 
conversion is required, at step S1819 it is performed. 
Then, at step S1820 the requested information is 
transmitted as a response and the processing is 

10 thereafter terminated. If the immediate transmission 
is not required, program control moves to step S1821, 
whereat a task for transmitting the data later is added 
to the task table. At step S1822 response = OK is 
issued, and the processing is terminated. 

15 Fig. 142 is a flowchart showing a response task 

relative to the request for storage of information. In 
this flowchart, a process will be explained that is 
initiated when the request type is a request for 
storage of information (step S1722 in Figs. 140A and 

20 140B). 

At step S1912 a check is performed to determine 
whether or not negotiation is required with the 
apparatus A that issued the request for an instruction. 
If the negotiation is required, program control moves 
25 to step S2013, whereat the negotiation is initiated. 

The object to be negotiated includes a data format for 
data execute and protocol/version exchange, and a data 
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format for other information required in advance for 
correct communication. This negotiation is repeated 
until the communication base used in common for the 
apparatuses A and B is established, 
5 Following this, at step S1914 information 

transmitted for the apparatus A is acquired. At step 
S1915 whether or not the information is received is 
determined. If the information is not yet received, 
program control goes to step S1916, whereat response = 

10 NG is returned, and the processing is terminated. If 
the information received, whereat a check is performed 
to determine whether or not data formatting or data 
conversion is required. If data formatting or data 
conversion is required, at step S1918 it is performed. 

15 Then, at step S1919 response = OK is issued, and the 
processing is terminated. 

Fig. 143 is a flowchart showing a response task 
relative to a request for an instruction. In this 
flowchart, a process will be explained that is 

20 initiated when the request type is a request for an 
instruction (step S1724 in Figs. 140A and 140B). 

At step S2012 a check is performed to determine 
whether or not negotiation is required with the 
apparatus A that issued the request for an instruction. 

25 If the negotiation is required, program control moves 
to step S1913, whereat the negotiation is initiated. 
The object to be negotiated includes a data format, 
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conversion protocol, upgrading of a version, 
adjustments associated with alteration of information 
types, and specification of the instruction contents. 
The negotiation process is planed before this process 
5 is initiated. 

Following this, at step S2014 instruction 
information is interpreted to decide a process to be 
performed next. At step S2015 a check is performed to 
determine whether or not the process can be executed. 

10 If the process can not be performed, response = NG is 
returned and the processing is terminated. 

When the process can be performed, program control 
moves to step S2017, whereat whether or not immediate 
execution is required. If the process need not be 

15 performed immediately, program control goes to step 

S2020, whereat a task for designating the performance 
of a corresponding process is added to the task table, 
and response = OK is returned. The processing is 
thereafter terminated . 

20 If the process should be immediately performed, 

program control advances to step S2018, whereat the 
instructed process is performed. The processing is 
thereafter terminated . 
[ Fifty-fourth Embodiment] 

25 An explanation of PULL task, for example pull 

printing for extracting and printing information, i.e., 
an example print on demand process will now be given, 



- 182 - 



by referring to Fig. 133, 

In the example in Fig. 133, a user sends an 
instruction 1333 to a printer A1331 to extract 
information 1334 "contract.doc" included in an 
5 apparatus B1332 that is located physically away from 

the printer A and to permit the printer A to print the 
information 1334. The apparatus B can be a printer, a 
scanner, a desktop PC, or a filing server. 

The contents "Please fetch 'contract.doc' from B 
10 and print it" which is an instruction from the user to 
the printer A1331, may be a command input using a 
mouse, or via a keyboard, by voice or a digitizer, or 
may be a sentence in the natural language by character 
recognition . 

15 When the apparatus B is a different device type 

from the apparatus A, e.g., a scanner, the information 
formatting is TIFF, BMP or JPEG, and format conversion 
is required. This data conversion may be performed 
either by the apparatus B or by the printer A to obtain 

20 the appropriate data format. When the apparatus B is a 
printer and information of the apparatus B has a format 
that the printer A can understand (e.g., when both 
apparatuses A and B are PCL printers ) , data conversion 
is not required. In other cases, even when both 

25 apparatuses are printers, data conversion is necessary. 
It should be noted that the apparatus A is not 
necessary a printer, and may be another output device, 
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such as a facsimile machine, a copier or a display. 

The transmission of information for the PULL task 
will now be described. Fig. 144 is a diagram showing 
an example pull session. In this example, the 
apparatuses A and B are printers, but as is described 
above, the apparatuses A and B need not necessarily be 
the same device type, and can be a scanner, a printer, 
a copier or a facsimile machine. 

The apparatus A performs the procedures in Figs. 
137A and 137B while the apparatus B performs the 
procedures in Fig. 141. First, the apparatus A 
contacts the apparatus B to establish a connection for 
communication (step S1412). When the response is OK 
(steps S1424 and S1822), a request for information 
(information <abc> in this example) and an information 
format, if necessary, are transmitted to the apparatus 
B ( step S1427 ) . 

The apparatus B fetches the requested information 
(step S1814), converts the format, as needed (step 
S1819), and transmits information in response to the 
request (step S1820). 

When the apparatus A extracts information from the 
apparatus B, it executes a necessary action in 
accordance with the instruction from the user or the 
determination of the system. The necessary action is, 
for example, printing, and in this case, pull printing 
is performed, i.e., information is extracted from a 
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remote area and printed. In other words, print on 
demand can be accomplished. 

Another necessary action is a display. In this 
case, pull viewing is performed, i.e., information is 
extracted from a remote area and displayed. In other 
words, view on demand can be accomplished. An 
additional necessary action is audio output. In this 
case, pull hearing is performed, i.e., information is 
extracted from a remote area and can be listened to. 
In other words, hear on demand can be accomplished. 

Similarly, further necessary actions are storage, 
filing and transferring. Any type of information can 
be extracted; a document, a file, electronic mail, a 
schedule, a status report, a log, voice mail or 
telephone call. 
[Fifty-fifth Embodiment] 

Fig. 145 is a diagram showing another example pull 
session. Along the same procedure as in Fig. 144, the 
apparatus A requests information <abc> to the apparatus 
B. However, the apparatus B does not immediately 
transmits the requested information (No at step S1817). 
A task for transmitting the data later is added to the 
task table (step S1821), and response = OK is returned 
to terminate the processing (step S1822). In this 
case, the transfer of the requested information is a 
print request from the apparatus B after a new session 
is established. 
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[Fifty-sixth Embodiment] 

Fig. 146 is a diagram showing an additional 
example pull session. In this example, a negotiation 
is held before the transfer of necessary information 
5 (steps S1425 and S1426 and steps S1812 to S1822). 

Since the information to be extracted by the apparatus 
A is a file, the apparatus A requests the apparatus B 
for a list of files. The apparatus B transmits the 
list of files to the apparatus A. From the list, the 

10 apparatus decides information necessary for the system 
or instructs the input of information necessary for a 
user. The apparatus A then decides the information to 
be requested, and issues a request for information to 
the apparatus B in the same manner as described above. 

15 [Fifty- seventh Embodiment] 

Fig. 147 is a diagram showing an example of Hear 
On Demand. In this example, "Read out New Mail 1 ' is 
input. While, in Fig. 147, an input window is employed 
for this input, any input device, such as a voice input 

20 device, a keyboard or a digitizer, can be employed as 
in the previous examples. 

In this example, the above command is transmitted 
to an apparatus A (printer) at Kosugi, and mail to be 
output is stored in an apparatus B (desktop PC) at 

25 Shimomaruko. The apparatuses A and B may be located in 
the same place, or in the same or different network 
domain. This session will be described while referring 
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to Fig, 148. 

Fig, 148 is a diagram showing a further example 
pull session. Information type is electronic mail and 
a requested format is voice. The procedures are, 
5 however, the same as previously described. 

The apparatus A requests the information (mail) 
(step S1427), the apparatus B converts the requested 
data into a suitable format (voice format in this 
example) (step S1819), and sends a response. Then, the 

10 apparatus A performs an optimal action. Since the 

optimal action is output of information by voice, the 
contents of mail is read aloud. While, in this 
example, the apparatus B converts text data into voice 
format, when the apparatus B can not convert the data 

15 the apparatus A may perform such conversion before the 
data output. 

[Fifty-eighth Embodiment] 

Fig. 149 is a diagram for explaining another pull 
task. In this example, an apparatus C is employed to 

20 issue a command to an apparatus A. The apparatus A is 
a display, and an instruction "Display my schedule" is 
issued to the apparatus C. Although the communication 
is not specified in the instruction, it means that a 
user extracts schedule data from the apparatus B 

25 (user's desktop PC) and displays it on the apparatus A. 

While the information type is schedule 
information, the procedures for this is the same as 
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explained previously. 

[ Fi f ty-ninth Embodiment ] 

Fig. 150 is a diagram for explaining an example 
push task. In this example, an apparatus A that 
5 receives a command is a scanner. The command shown in 
Fig. 150 is issued to instruct the apparatus A to 
transmit a document obtained by scanning to an 
apparatus B (printer) and to instruct the apparatus B 
to hold the document instead of printing it. This 
10 session will now be described. 

Fig. 151 is a diagram showing a session for the 
push task in Fig. 150. As is described above, the 
apparatus A is a scanner and the apparatus B is a 
printer. 

15 The apparatus A performs the procedures in Figs. 

138A and 138B while the apparatus B performs the 
procedures in Fig. 132. First, the apparatus A 
contacts the apparatus B to establish a connection for 
communication (step S1512). Assume that the response 

20 is OK (steps S1524 and S1718). 

Then, transfer of information (information <abc> ) 
is requested. Since the request type is a request 
storage of information, the apparatus performs the 
procedures in Fig. 142. At the negotiation (step S1526 

25 and S1913), the apparatus B requests the apparatus A 

for a format ( <xyz> ) of information to be transferred. 
The apparatus A changes the format, if necessary, 
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and transmits the requested information (step S1527). 
Upon the receipt of the information, the apparatus B 
performs a required action according to the decision of 
the transmission side, the user or the apparatus B. 
5 The action is, for example, format conversion, filing, 
index preparation, data extraction, or title 
extraction. The necessary action in this embodiment is 
to hold a received document instead of printing it. 
O [Sixtieth Embodiment] 

J3 10 Fig. 152 is a diagram showing another example push 

fQ task. The apparatus A, which is a scanner in this 

rt example, sends a request to the apparatus B (scanner) 

^ and the apparatus C (filing server) to push 

[7; information, to file a document on a reception side and 

15 to extract its title and index. 
?S The specific processing is the same as that in the 

fifty-ninth embodiment. The session between the 
apparatuses A and B will now be described while 
referring to Fig. 153. The same thing is applied for 
20 the session between the apparatuses A and C. 

Communication between the apparatuses A and B and 
communication between the apparatuses A and C are 
performed in parallel. 

Fig. 153 is a diagram showing an example push 
25 session. The apparatuses A and B are scanners. when 
the apparatus B receives data, it performs a required 
action. In this case, the apparatus A transmits 
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information for designating an associated action that 
should be executed. The action is filing and 
extraction of a title and an index. While in this 
example the action is instructed from the apparatus A, 
5 the apparatus B may decide an action, and may abandon a 
received action. 
[ S ix ty- first Embodiment ] 

Fig. 154 is a diagram showing another example push 
task, which is a transmission of electronic mail. An 

10 apparatus A (printer) at Kosugi is to transmit 

electronic mail "Get design from Bill" to an apparatus 
B (having a different network sub-domain from John's PC 
and the apparatus A) in New York. 

Any method can be used for mail transmission. 

15 First, when the apparatus A has a mail transmission 
function, it transmits the mail. If the apparatus A 
does not have such a function, it transmits mail 
information to another apparatus on the network, e.g., 
a mail server, and asks the mail transmission. 

20 [Sixty-second Embodiment] 

The instruction task INDICATE will be described 
while referring to Fig. 132. A user sends an 
instruction 93 to a copier 91 that is physically 
located in a remote area in order to permit a MyDesk 

25 92, a desktop personal computer, to display the current 
location of the user. 

The instruction 93 "Display present place on 
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<MyDesk> !! , which is sent to the copier 91 that serves 
as the information processing apparatus 411, is 
transmitted from the operation unit 41 to the task 
reception unit 43 . The contents are analyzed and 
planning is performed on the results. Then, the 
execution unit 47 executes the final data as an 
instruction task INDICATE for a PC 92, which serves as 
the information processing apparatus 412 connected via 
the network . 

The PC 92, which is the information processing 
apparatus 412 connected via the network, receives 
instructed information from the reception unit 43, 
analyzes a received instruction, and perform planning 
on the results. Then, the execution unit 47 displays 
the location, such as "Now in conference Shimomaruko 
Alll conference room" on a screen. 

Transmission of information for each session 
between the information processing apparatuses 411 and 
412 will be explained while referring to Fig. 155. 
Fig. 155 is a diagram showing transmission of 
information for each session between a copier, which 
serves as the information processing apparatus 411 for 
instruction task INDICATE, and a PC, which serves as 
the information processing apparatus 412. 

The transmission side performs the processing in 
Figs. 139A and 139B, while the reception side performs 
the processing in Fig. 143. The communication contents 
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are an instruction and information for processing 
corresponding to the instruction. The transmission of 
information is the same as that in the above described 
push task. 

5 In response to the instruction request from the 

apparatus A, the apparatus B transmits <xyz> 
instruction type that can be understood by the 
apparatus B. Upon receipt of the information, the 

p apparatus A changes the instruction information, as 

g3 10 needed. 

in Then, the apparatus A issues, to the apparatus B, 

hj an instruction and necessary information to display on 

UJ 

the PC screen the fact that a conference is being held 
p U and the current location. The apparatus B receives the 

15 information, performs the process in consonance with 

the information, and displays the location of the user 
on the screen, as is instructed. 
[ Sixty- third Embodiment ] 

Another example instruction task INDICATE will now 
20 be described while referring to Fig. 130. A user sends 
an instruction 71 to a copier 74 at a physically remote 
area in order to set a facsimile machine 75 at 
Shinkawasaki so that it transfers, to a printer 77 of 
Mr. Toyoda at Kosugi, a document that is received from 
25 Mr. Suzuki. 

The instruction 71 "Please advice MyFax to 
transfer any document sent from Mr. Suzuki to Mr. 
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Toyoda" which is sent: to the copier 74 that serves as 
the information processing apparatus 411, is 
transmitted from the operation unit 41 to the task 
reception unit 43 . The contents are analyzed and 
5 planning is performed on the results. Then, the 
execution unit 47 executes the final data as an 
instruction task INDICATE for the facsimile machine 75, 
which serves as the information processing apparatus 
O 412 connected via the network. 

■ % q 10 The facsimile machine 75, which is the information 

gg processing apparatus 412 connected via the network, 

hj receives instruction information from the reception 

1~ unit 43, analyzes a received instruction, and perform 

|t =5: 

pi i planning on the results. Then, the execution unit 47 

; 2 15 alters the condition setting. The transmission of 

^ information for each session between the information 

processing apparatuses 411 and 412 will be described 
while referring to Fig. 156. 

Fig. 156 is a diagram showing transmission of 
20 information for each session between a copier, which 
serves as the information processing apparatus 411 in 
this example instruction task INDICATE, and a facsimile 
machine, which swerves as the information processing 
apparatus 412. The transmission side performs the 
25 processing in Figs. 139A and 139B, while the reception 
side performs the processing in Fig. 143. The 
transmission of information is the same as that in the 
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above described example . 

In response to the instruction request from the 
apparatus A, the apparatus B transmits <xyz> 
instruction type that can be understood by the 
5 apparatus B. Upon receipt of the information, the 
apparatus A changes the instruction information, as 
needed. In this example, the apparatus A transmits to 
the apparatus B an instruction and necessary 
information to transfer, to Mr. Toyoda, a document that 

10 is transmitted from the Mr. Suzuki and addresses the 

user. Upon receipt of the information, the apparatus B 
performs the required processing in consonance with the 
information, and sets a designated condition. 
[Sixty- fourth Embodiment] 

15 An additional example instruction task INDICATE 

will now be described while referring to Fig. 135. A 
user sends an instruction 123 to a scanner 121 at a 
physically remote area in order to instruct a filing 
server 122 to transfer a stored document 125 "abc.doc" 

20 to John 124. 

The instruction 123 "Action: SEND, What: r, From: 
Filing Server, To: John" which is sent to the scanner 
121 that serves as the information processing apparatus 
411, is transmitted from the operation unit 41 to the 

25 task reception unit 43. The contents are analyzed and 
planning is performed on the results. Then, the 
execution unit 47 executes the final data as an 
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instruction task INDICATE for the filing server 122, 
which serves as the information processing apparatus 
412 connected via the network. 

The filing server 122, which is the information 
5 processing apparatus 412 connected via the network, 
receives instruction information from the reception 
unit 43, analyzes a received instruction, and perform 
planning on the results. Then, the execution unit 47 
C3 transmits a document. The transmission of information 

10 for each session between the information processing 
fO apparatuses 411 and 412 will be described while 

hj referring to Fig. 157. 

: • : 

^ Fig. 157 is a diagram showing transmission of 

ire 

~ !S information for each session between a scanner, which 

iU 

f ! £ 

15 serves as the information processing apparatus 411, and 
j~ a filing server, which serves as the information 

processing apparatus 412. The transmission side 
performs the processing in Figs. 139A and 139B, while 
the reception side performs the processing in Fig. 143. 

20 The transmission of information is the same as that in 
the above described example. 

In response to the instruction request from the 
apparatus A, the apparatus B transmits <xyz> 
instruction type that can be understood by the 

25 apparatus B. Upon receipt of the information, the 
apparatus A changes the instruction information, as 
needed. In this example, the apparatus A transmits to 
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•the apparatus B an instruction and necessary 
information to transfer, to John, a document "abc.doc" 
that is stored in the filing server* 

Upon receipt of the information, the apparatus B 
5 performs the required processing in consonance with the 
information, decides an optimal transmission method, 
such as E-mail, and transmits the document to John as 
is instructed. 
[ Sixty- f i f t h Embodiment ] 

10 Fig. 158 is a diagram showing an operation along 

the procedures of another apparatus. In the sixty- 
fourth embodiment explained while referring to Fig. 
135, a user sends an instruction to the scanner 121 
that is located at a physically remote area, in order 

15 to instruct the filing server 122 to transfer the 

stored document 125 "abc.doc" to the John's desktop 
124. And the scanner 121 interprets contents and 
executes the instruction task INDICATE. 

In this embodiment, however, a printer 351 

20 receives operation information 353 from a filing server 
352 and displays it, and a user performs operation in 
accordance with the display and transmits the operation 
data to the filing server 352. In this manner, the 
same effect as in the sixty- fourth embodiment can be 

25 obtained. 

When a user requests the printer 351, which serves 
as the information processing apparatus 411, to operate 
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-the filing server 352, a planning decision unit 46 in 
•the printer 351 makes a plan that necessary operation 
data 353 should be obtained to implement the operation 
procedures of the filing server 352. The process 
5 execution unit 47 executes the information acquisition 
task PULL for the filing server 352, which is the 
information processing apparatus 412 connected via the 
network . 

£3 The filing server 352, which is the information 

y3 10 processing apparatus 412 connected via the network, 

£0 receives instruction request information from the 

id reception unit 43, analyzes a received request, and 

s perform planning on the results. Then, the execution 

fy unit 47 transmits the operation data 353. 

F! I 

'J* 15 A screen is provided on the operation unit 41 of 

^ the scanner 351, which receives the operation data 353 

from the filing server 352, and the operation unit 41 
displays, on the screen, the procedures of the filing 
server 352 that correspond to the operation data 353. 
20 According to the displayed operation screen, the 

information concerning the operation of the user is 
transmitted to the filing server 352 in the same 
manner. The filing server 352 that receives the 
operation of the user interprets it and performs a 
25 necessary process. As a result, transmission of the 
file " abc.doc" to John, which is the object to the 
user, is enabled. 
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In the above description, the filing server 352 
transmits the operation data 353 for implementing the 
operation procedures of the filing server 352; however, 
an operation data server may be prepared for managing 
5 the operation data 353, and may transmit the operation 
data 353. The operation data may also be transmitted 
from the third information processing apparatus that 
receives the operation data 353 of the filing server 
p 352, like the printer 351 that serves as the 

tQ 10 information processing apparatus 411. The printer 351 

tjj may be replaced with a scanner. 

[ Sixty- sixth Embodiment ] 

Fig. 162 is a diagram showing the relationship 
between the change to three statuses of a print job and 
15 processes corresponding to the individual statuses. An 
output device (printer), which serves as the 
information processing apparatus 411, receives a print 
job that includes a document for which a user 
instructed printing . 
20 First, an input job determination unit 391 

determines whether a received print job should be 
immediately printed or should be held until a print 
execution condition is established. The result is 
transmitted to a pending job management unit 392 or an 
25 output wait job management unit 394. 

The pending job management unit 392 stores the 
received print job as a pending job in a pending job 
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table 393. The pending job management unit 392 
examines the pending job table 393 for the job for 
which print execution condition is established. If 
such a job is found, it is extracted from the pending 
5 job table 393. The pending job for which the print 

condition is established is transmitted to the output 
wait job management unit 394. 

The output wait job management unit 394 stores a 
received print job as an output wait job in an output 

10 wait job table 395. When printing at the output unit 
396 is enabled, the output wait job is extracted from 
the output wait job table 395 and transmitted to the 
output unit 396. 

The output unit 39 6 prints the received job. When 

15 printing is successful, the print job is transmitted to 
an output job management unit 397 to store the print 
job that has been output. 

The output job management unit 397 stores, as an 
output job, the received print job that has been output 

20 in an output job table 398. 

As is described above, the received print job is 
changed into three statues and processed. The print 
execution condition of a pending job is not only a 
condition where the job is held until the user 

25 instructs printing, but also a condition where printing 
is initiated at a designated time, or a condition 
where, if the main printer is other than printer A that 
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serves as "the information processing apparatus 411, the 
main printer is ready for printing* 

Figs. 163A and 163B are flowcharts showing the 
printing processing for this embodiment. The 
processing of a print task "PRINT (A, 'abc.doc' )" or 
"PRINT (not A, 'abc.doc') 11 is performed when an 
instruction is issued to the printer A, which is the 
information processing apparatus 411, to print document 
"abc.doc" by the printer A and by a printer other than 
the printer A. 

When the print task PRINT is activated, at step 
S4001 a check is performed to determine whether or not 
a document to be printed is included in a print job. 
When there is no document to be printed, program 
control moves to step S4002. At step S4002 a check is 
performed to determine whether the document is included 
among the documents in the print jobs that are entered 
in the pending job table 393, the output wait job table 
395 or the output job table 398. When, at step S4003, 
the document is not found, program control advances to 
step S4004. 

At step S4004 the information acquisition task 
PULL is performed in order to acquire the document from 
another information processing apparatus 412. If, at 
step S4005, it is ascertained that the document is 
successfully acquired, program control advances to step 
S4006. If the acquisition of the document fails, 
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program control moves "to step S4016, whereat an error 
is displayed and another necessary process is 
performed. The processing is thereafter terminated. 
At step S4006 a check is performed to determine whether 
5 the document to be printed can be printed by the 
printer A, which is the information processing 
apparatus 411, or by a printer other than the printer 
A. If the conversion of the document into a printer 

£3 description language is required, program control moves 

%n 10 to step S4007. 

£0 At step S4007 a check is performed to determine 

|7| whether or not the printer A, which is the information 

Lii 

processing apparatus 411, can convert the document 
fis form. If the conversion is possible, program control 

*2 15 goes to step S4008, whereat the document to be printed 

is converted into a suitable form. When the conversion 
is impossible, program control goes to step S4009, 
whereat instruction INDICATE is issued to another 
information processing apparatus 412 to convert the 
20 document to be printed. According to the instruction 
INDICATE, the information processing apparatus 412 may 
immediately perform the conversion and return the 
result, or may return only an acknowledgement and 
perform conversion after printing is instructed later. 
25 At step S4010 a check is performed to determine 

whether or not the conversion result is successful. If 
it is successful, program control advances to step 
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S4011. If the conversion fails, program control moves 
to step S4016, whereat an error is displayed and 
another required process is performed. The processing 
is thereafter terminated. 
5 At step S4011 a check is performed to determine 

whether or not the printer A, which is the information 
processing apparatus 411, should print a document. 
When it is determined the printer A should print the 
document, program control advances to step S4012, and 

10 the document is printed. When the information 

processing apparatus 412 should print the document, 
program control goes to step S4013, and the information 
storage task PUSH and print task PRINT for the 
information processing apparatus 412 are performed. 

15 At step S4014, whether or not the printing is 

successful is determined. When the printing is 
successful, program control advances to step S4015, and 
the print job is regarded as the output job, which is 
then stored in the output job table 398. The other 

20 necessary process is performed, and the processing is 
terminated. When the printing fails, program control 
moves to step S4016, whereat an error is displayed and 
another required process is performed. The processing 
is thereafter terminated. 

25 The conversion of the document at step S4008 

includes a conversion of a normal text document into a 
document in a printer description language; a 
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conversion of a document in a specific printer language 
into another printer language that the employed printer 
can understand; a conversion of an image format 
document read by a scanner or a digital camera into a 
5 document in a printer language; or a conversion of a 
document in the common language in WWW into a printer 
language. 

The conversion instruction INDICATE at step S4009 
may be an instruction to the information processing 

10 apparatus 412 that has a specific application to 
convert a document in a language inherent to the 
corresponding application into a document in a printer 
language that can be printed; or an instruction to the 
information processing apparatus 412 that has a 

15 specific printing function to convert a document in a 

language inherent to a specific printer into a document 
in a printer language that can be printed. 

Fig. 164 is a diagram showing the relationship 
between the print job and the processing, which are 

20 explained while referring to Figs. 162 and 163, and the 
user's operation. When a user permits the scanner 
A411, which serves as the information processing 
apparatus 412, to read document "abc.doc" 144 and 
issues an instruction "Send 'abc.doc' to printer B" the 

25 printer B 412, which serves as the information 

processing apparatus 411 that receives the print job, 
does not print "abc.doc" immediately, but regards it as 
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a pending job and stores it: in the pending job table 
393. 

When the user sends an instruction "Show pending 
documents" the document "abc.doc" is displayed on the 
5 operation panel of the printer B. Furthermore, when 
the user designates the document "abc.doc" and issues 
an erasure instruction "Delete pending document" the 
designated document is deleted from the pending job 
table 393. 

10 When the user designates the document "abc.doc" 

and issues an instruction "Print pending document" the 
document "abc.doc" is changed to a print wait job, 
which is then stored in the output wait job table 395. 
When the printing is enabled, it is printed (415). 

15 When the printing is successful, the document 

"abc.doc" is changed to the output job, which is then 
stored in the output job table 395. In addition, when 
the user designates the output document "abc.doc" and 
issues an erasure instruction "Delete printed document" 

20 the designated document is deleted from the output job 
table 398. 

Fig. 165 is a diagram showing an example display 
screen that is used by a user to operate a print job in 
the three statues as explained in Fig. 162. In 
25 accordance with the operation by a user, an instruction 
from the information processing apparatus 412, or the 
result obtained by the process performed by the 
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information processing apparatus 411, the print jobs in 
the three statuses are operated arbitrarily, and are 
added as a new task to the task table 399. 

For example, a user utilizes the operation screen 
5 in Fig. 165 to delete (4209) a print job stored in the 
pending job table 393 (4201), to send (4208) the print 
jot to print using another printer, or to print it 
immediately. It is also possible to instruct to 
increase the priority (4210) of a print job stored in 
10 the output wait job table 395 (4202) so that it will be 
printed first, or a print job, or to print (4204) again 
the print job stored in the output job table 398 
(4203) . 

The present invention can be applied either for a 
15 system that is constructed by a plurality of 

apparatuses (e.g., a host computer, an interface 
device, a display) or for an apparatus including a 
single device. 

The following is also included within the scope of 
20 the present invention: in order to operate various 
devices to implement functions in the above 
embodiments, software program code for implementing the 
functions in the previous embodiments are supplied to a 
computer in an apparatus or in a system that is 
25 connected to various devices, and in consonance with 
the program, the computer (or a CPU or an MPU) in the 
system or the apparatus operates the devices to 
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accomplish "the functions in the above embodiments. In 
this case, the program code read from a memory medium 
accomplishes the functions of the above described 
embodiments. And the program code and means for 
5 supplying the program code to the computer, e.g., a 

memory medium on which such program code is recorded, 
constitute the present invention. 

A memory medium for supplying such program code 
can be, for example, a floppy disk, a hard disk, an 

10 optical disk, a magneto optical disk, a CD-ROM, a CD-R , 
a magnetic tape, a nonvolatile memory card, or a ROM. 

In addition, not only for a case where the 
functions in the previous embodiments can be performed 
when program code is read and executed by the computer, 

15 but also for a case where, according to an instruction 
in the program code, an OS (Operating System) running 
on the computer, or another application software 
program, interacts with the program code to accomplish 
the functions in the above embodiments, this program 

20 code can be included in the embodiments of the present 
invention. 

Furthermore, the present invention includes a case 
where program code, read from a memory medium, is 
written in a memory that is mounted on a function 
25 expansion board inserted into a computer, or in a 

function expansion unit connected to a computer, and in 
consonance with a program code instruction, a CPU 
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mounted on the function expansion board or the function 
expansion unit performs one part, or all of the actual 
processing in order to implement the functions in the 
above described embodiments. 

When the present invention is applied for the 
above memory medium, program code corresponding to the 
previously described flowcharts is stored in the memory 
medium. 



